-- Logs begin at Thu 2022-09-01 15:48:31 UTC, end at Thu 2022-09-01 16:52:03 UTC. -- Sep 01 16:19:49.335148 dv-cinder-852209-10-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 01 16:19:49.346411 dv-cinder-852209-10-iscsi devstack@keystone.service[81538]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 01 16:19:49.348466 dv-cinder-852209-10-iscsi devstack@keystone.service[81538]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 1 16:19:49 2022] *** Sep 01 16:19:49.348466 dv-cinder-852209-10-iscsi devstack@keystone.service[81538]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 01 16:19:49.348466 dv-cinder-852209-10-iscsi devstack@keystone.service[81538]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 01 16:19:49.348466 dv-cinder-852209-10-iscsi devstack@keystone.service[81538]: nodename: dv-cinder-852209-10-iscsi Sep 01 16:19:49.348466 dv-cinder-852209-10-iscsi devstack@keystone.service[81538]: machine: x86_64 Sep 01 16:19:49.348466 dv-cinder-852209-10-iscsi devstack@keystone.service[81538]: clock source: unix Sep 01 16:19:49.348466 dv-cinder-852209-10-iscsi devstack@keystone.service[81538]: pcre jit disabled Sep 01 16:19:49.348466 dv-cinder-852209-10-iscsi devstack@keystone.service[81538]: detected number of CPU cores: 4 Sep 01 16:19:49.348466 dv-cinder-852209-10-iscsi devstack@keystone.service[81538]: current working directory: / Sep 01 16:19:49.348466 dv-cinder-852209-10-iscsi devstack@keystone.service[81538]: detected binary path: /usr/bin/uwsgi-core Sep 01 16:19:49.348466 dv-cinder-852209-10-iscsi devstack@keystone.service[81538]: your processes number limit is 47680 Sep 01 16:19:49.348466 dv-cinder-852209-10-iscsi devstack@keystone.service[81538]: your memory page size is 4096 bytes Sep 01 16:19:49.349074 dv-cinder-852209-10-iscsi devstack@keystone.service[81538]: detected max file descriptor number: 2048 Sep 01 16:19:49.349074 dv-cinder-852209-10-iscsi devstack@keystone.service[81538]: lock engine: pthread robust mutexes Sep 01 16:19:49.349074 dv-cinder-852209-10-iscsi devstack@keystone.service[81538]: thunder lock: enabled Sep 01 16:19:49.349074 dv-cinder-852209-10-iscsi devstack@keystone.service[81538]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 01 16:19:49.349074 dv-cinder-852209-10-iscsi devstack@keystone.service[81538]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 01 16:19:49.388303 dv-cinder-852209-10-iscsi devstack@keystone.service[81538]: Python main interpreter initialized at 0x55d5575939c0 Sep 01 16:19:49.388303 dv-cinder-852209-10-iscsi devstack@keystone.service[81538]: python threads support enabled Sep 01 16:19:49.388303 dv-cinder-852209-10-iscsi devstack@keystone.service[81538]: your server socket listen backlog is limited to 100 connections Sep 01 16:19:49.388560 dv-cinder-852209-10-iscsi devstack@keystone.service[81538]: your mercy for graceful operations on workers is 90 seconds Sep 01 16:19:49.388641 dv-cinder-852209-10-iscsi devstack@keystone.service[81538]: mapped 403077 bytes (393 KB) for 2 cores Sep 01 16:19:49.388716 dv-cinder-852209-10-iscsi devstack@keystone.service[81538]: *** Operational MODE: preforking *** Sep 01 16:19:49.388992 dv-cinder-852209-10-iscsi devstack@keystone.service[81538]: *** uWSGI is running in multiple interpreter mode *** Sep 01 16:19:49.388992 dv-cinder-852209-10-iscsi devstack@keystone.service[81538]: spawned uWSGI master process (pid: 81538) Sep 01 16:19:49.389192 dv-cinder-852209-10-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 01 16:19:49.390695 dv-cinder-852209-10-iscsi devstack@keystone.service[81538]: spawned uWSGI worker 1 (pid: 81551, cores: 1) Sep 01 16:19:49.390695 dv-cinder-852209-10-iscsi devstack@keystone.service[81538]: spawned uWSGI worker 2 (pid: 81552, cores: 1) Sep 01 16:19:49.390695 dv-cinder-852209-10-iscsi devstack@keystone.service[81538]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 01 16:19:53.653237 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 16:19:53.653237 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: from cryptography.utils import int_from_bytes Sep 01 16:19:53.653999 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 16:19:53.653999 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: from cryptography.utils import int_from_bytes Sep 01 16:19:53.654455 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 16:19:53.654455 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: from cryptography.utils import int_from_bytes Sep 01 16:19:53.654561 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 16:19:53.654561 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: from cryptography.utils import int_from_bytes Sep 01 16:19:53.702907 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 16:19:53.703104 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 16:19:53.703300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] command line args: [] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 16:19:53.703433 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 16:19:53.703576 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 16:19:53.703662 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 16:19:53.703748 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ================================================================================ {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 16:19:53.703819 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] command line args: [] {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 16:19:53.704003 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] admin_token = **** {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.704603 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 16:19:53.704764 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] config_dir = [] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.704852 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ================================================================================ {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 16:19:53.704955 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.705039 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] admin_token = **** {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.705140 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] config_source = [] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.705275 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] config_dir = [] {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.705358 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] debug = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.705468 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.705552 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.706867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] config_source = [] {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.706867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] debug = True {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.707065 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.707156 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.707257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.707329 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.707445 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.707558 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.707680 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.707802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.707917 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] list_limit = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.708034 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.708147 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] log_config_append = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.708147 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.708372 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] list_limit = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.708474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] log_dir = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.708590 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] log_config_append = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.708708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] log_file = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.708822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.708933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.709052 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] log_dir = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.709052 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] log_file = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.709251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.709362 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.709480 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.709592 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.709715 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.709715 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.709940 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.710058 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.710178 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.710320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.710442 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.710442 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.710650 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.710769 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.710890 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.711001 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.711114 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.711232 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.711344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.711462 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.711574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.711694 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.711813 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.711922 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.712045 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.712163 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.712281 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.712403 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.712403 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.712627 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.712627 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] publish_errors = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.712627 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.712951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.713071 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.713189 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] publish_errors = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.713309 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.713422 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.713533 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.713644 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.713754 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.713883 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.714004 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] standard_threads = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.714124 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.714251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.714359 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.714483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.714603 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] standard_threads = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.714716 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.714834 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.714954 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] use_journal = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.715068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.715234 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] use_json = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.715355 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.715475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] use_stderr = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.715588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] use_journal = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.715706 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] use_syslog = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.715819 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] use_json = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.715939 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.716059 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] use_stderr = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.716172 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.716284 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] use_syslog = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.716395 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.716513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:19:53.716636 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.716750 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.716882 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.716990 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.717109 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.717222 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.717342 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.717461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.717581 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.717699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.717818 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.717930 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.718051 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.718164 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.718304 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.718427 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.718541 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.718653 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.718776 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.718890 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.719008 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.719121 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.719233 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.719352 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.719473 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.719591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.720364 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.720491 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.720605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.720723 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.720846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.720960 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.721072 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.721185 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.721297 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.721487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.721610 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.721724 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.721836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.721947 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.722059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.722187 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.722330 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.722446 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.722558 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.722671 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.722782 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.722903 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.723016 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.723128 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.723246 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.723359 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.723471 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.723591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.723704 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.723816 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.723927 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.724039 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.724151 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.724275 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.724388 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.724500 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.724619 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.724731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.724843 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.724969 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.725083 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.725195 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.725307 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.725424 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.725536 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.725660 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.725774 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.725886 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.725998 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.726110 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.726221 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.726413 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.726528 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.726641 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.726759 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.726871 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.726983 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.727170 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.727286 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.727398 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.727510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.727621 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.727733 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.727857 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.727971 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.728083 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.728195 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.728307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.728424 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.728548 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.728661 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.728758 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.728857 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.728952 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.729045 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.729149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.729243 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.729337 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.729439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.729533 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.729626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.729728 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.729849 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.729962 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.730107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.730229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.730372 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.730488 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.730600 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.730713 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.730826 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.730940 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.731063 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.731177 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.731295 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.731407 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.731520 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.731639 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.731750 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.731865 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.731989 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.732113 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.732195 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.732195 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.732410 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.732410 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.732610 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.732724 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.732837 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.732940 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.733059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.733173 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.733286 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.733406 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.733520 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.733520 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.733709 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.733826 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.733948 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.734068 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.734183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.734343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.734447 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.734559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.734683 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.734799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.734918 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.735033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.735145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.735301 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.735430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.735535 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.735654 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.735769 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.735882 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.736001 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.736125 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.736240 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.736353 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.736466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.736579 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.736692 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.736813 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.736934 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.737048 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.737163 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.737276 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.737389 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.737511 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.737626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.737739 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.737852 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.737966 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.738078 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.738287 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.738411 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.738525 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.738639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.738752 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.738874 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.738996 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.739119 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.739235 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.739349 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.739462 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.739576 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.739689 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.739813 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.739929 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.740042 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.740155 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.740269 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.740383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.740508 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.740623 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.740750 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.740866 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.740992 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.741106 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.741309 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.741429 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.741543 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.741657 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.741770 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] auth.external = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.741891 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.742035 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.742204 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.742375 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.742494 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.742607 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.742721 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] auth.external = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.742841 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] auth.password = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.742954 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.743076 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] auth.token = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.743076 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.743290 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.743404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.743517 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.743629 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.743751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] auth.password = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.743864 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.743983 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] auth.token = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.744096 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.744210 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.744322 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.744444 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.744558 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.744671 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.744783 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] credential.caching = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.744904 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.745018 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.745152 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.745269 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.745383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.745497 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.745611 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.745731 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.745854 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] credential.caching = True {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.745968 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.746082 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.746202 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.746334 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.746450 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.746578 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.746694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.746808 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.746922 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.747037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.747156 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.747280 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.747395 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.747509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.747623 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.747744 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.747858 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.747981 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.748096 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.748210 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.748323 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.748441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.748555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] federation.caching = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.748678 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.748793 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.748919 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.749035 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.749155 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.749278 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.749393 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.749507 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.749621 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] federation.caching = True {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.749736 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.749857 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.749980 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.750095 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.750209 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.750342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.750456 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.750570 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.750696 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.750834 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.751226 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.751341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.751455 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.751569 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.751691 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity.caching = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.751813 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.751927 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.752041 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.752155 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.752270 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.752460 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.752579 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.752694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.752808 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] identity.caching = True {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.752923 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.753038 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.753163 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.753278 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.753392 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.753506 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.753620 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.753734 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.753856 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.753972 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.754086 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.754190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.754319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.754417 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.754543 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.754543 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.754773 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.754888 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.755003 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.755003 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.755224 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.755339 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.755453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.755567 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.755670 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.755783 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.755908 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.756024 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.756138 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.756251 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.756365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.756479 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.756602 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.756724 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.756839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.756839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.757054 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.757157 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.757278 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.757393 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.757507 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.757620 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.757734 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.757848 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.757970 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.758086 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.758200 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.758376 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.758494 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.758617 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.758732 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.758846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.758960 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.759073 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.759188 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.759188 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.759449 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.759449 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.759664 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.759777 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.759891 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.760016 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.760133 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.760248 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.760362 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.760476 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.760590 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.760711 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.760826 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.760940 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.761054 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.761174 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.761290 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.761411 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.761526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.761641 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.761755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.761869 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.761983 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.762104 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.762219 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.762355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.762470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.762583 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.762707 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.762831 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.762946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.763066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.763181 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.763294 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.763409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.763530 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.763646 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.763760 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.763873 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.763987 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.764101 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.764222 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.764337 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.764451 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.764564 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.764678 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.764791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.user = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.764913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.765029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.765142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.765256 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.765369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.765483 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.765605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.765720 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.765834 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.765948 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.766061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.766175 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.766314 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.766432 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.767029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.767146 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.767268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.767383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.767505 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.767621 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.767735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.767849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.767963 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.768077 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.768199 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.768315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.768434 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.768548 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.768662 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.768776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.768898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.769014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.769128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.769242 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.769355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.769469 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.769592 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.769708 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.769822 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.769936 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.770050 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.770164 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.770336 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.770455 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.770569 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.770684 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.770797 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.770918 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.771041 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.771157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.771271 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.771385 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.771499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.771612 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.771738 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.771854 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.771967 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.772082 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.772195 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.772308 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.772431 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.772545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.772660 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] resource.caching = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.772775 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.772888 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.773003 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.773125 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.773240 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.773354 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.773468 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.773581 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.773695 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.773817 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.773933 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] resource.caching = True {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.774047 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.774160 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.774287 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.774416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.774532 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.774646 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.774761 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.774876 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.774990 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] role.caching = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.775177 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.775295 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] role.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.775409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.775523 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.775637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.775751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.775873 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.775990 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.776104 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.776218 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.776332 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] role.caching = True {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.776445 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.776567 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] role.driver = sql {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.776681 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.776796 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.776909 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.777025 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.777139 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.777261 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.777376 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.777491 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.777605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.777718 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.777832 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.777954 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.778075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.778190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.778323 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.778512 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.778630 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.778758 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.778758 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.778974 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.779089 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.779203 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.779317 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.779439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.779554 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.779668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.779781 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.779895 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.779895 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.780119 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.780234 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.780348 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.780462 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.780576 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.780702 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.780818 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.780931 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.781044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.781159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.781272 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.781394 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.781509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.781623 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.781737 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.781851 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.781965 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.782086 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.782892 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.783017 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.783132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.783246 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.783362 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.783489 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.783606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.783721 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.783835 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.783949 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] token.caching = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.784062 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.784184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.784300 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.784414 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.784528 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.784642 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.784755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.784877 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.784993 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.785106 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.785221 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] token.caching = True {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.785335 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.785449 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.785572 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.785687 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.785800 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.785914 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.786028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.786141 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.786348 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.786469 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.786583 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.786697 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.786811 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.786925 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.787047 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.787163 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.787276 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.787381 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.787495 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.787618 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.787746 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.787870 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.787985 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.788100 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.788213 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.connection = **** {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.788328 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.788446 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.788571 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.788687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.788801 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.788924 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.789048 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.789163 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.789278 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.789392 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.789392 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.789618 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] database.connection = **** {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.789738 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.789853 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.789967 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.790081 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.790203 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.790331 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.790446 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.790560 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.790675 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.790789 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.790914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.791030 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.791143 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.791256 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.791369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.791483 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.791604 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.791720 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.791834 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.791947 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.792061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.792174 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.792296 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.792410 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.792524 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81551) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 16:19:53.792646 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.792646 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.792646 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.792646 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.792646 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.792646 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:19:53.792646 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81552) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 16:19:53.819683 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:53.819921 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:53.820060 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81552) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:53.820145 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:53.820300 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81552) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:53.820533 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81552) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:53.828408 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:53.829516 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81552) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:53.857113 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:53.857348 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:53.857431 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81552) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:53.857610 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81552) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:53.910817 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:53.911242 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81552) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:53.958850 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:53.959087 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:53.960994 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81552) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:53.961241 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81552) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.113971 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.114193 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.114448 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.114677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.114900 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.115119 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.115268 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81552) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.115511 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81552) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.115743 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81552) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.115967 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81552) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.116190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81552) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.116411 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81552) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.138833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.138978 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81552) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.139063 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.139221 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81552) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.139304 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.139454 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81552) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.139535 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.139684 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81552) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.139775 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.139900 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.139981 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81552) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.140117 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.140198 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81552) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.140337 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81552) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.151508 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.151997 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81552) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.164347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.164453 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81552) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.164534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.164701 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81552) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.164783 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.164938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81552) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.165020 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.165169 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81552) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.165259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.165384 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.165464 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81552) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.165601 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81551) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.165682 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81552) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.165822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81552) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:19:54.182306 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.182523 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 16:19:54.182761 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.182870 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.182978 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 16:19:54.183075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 16:19:54.183339 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.183565 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.183646 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.183775 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 16:19:54.183865 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.184063 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.184143 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.184335 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.184604 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.184873 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.194187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.194486 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.194762 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.194842 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.195046 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.195138 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.195323 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.195614 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.201944 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.202580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81552) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.205555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.205912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.206055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81552) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.206326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.206403 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.206625 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.206711 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.206915 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.206986 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.207183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.207253 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.207458 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.207529 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.207791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.207869 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.208180 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.208259 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.208519 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.208624 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.208883 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.209078 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.209261 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.232875 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.234416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81552) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.234713 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.236794 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.238311 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.238603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.239426 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81552) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.239721 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.240015 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.243147 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.243437 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.243727 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.246397 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81552) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.246692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.250220 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.250531 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.250811 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81552) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.251884 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.252174 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.252462 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.252742 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.253025 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.253299 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.253570 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.255369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.255666 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.255957 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.256248 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.256530 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.258676 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.258973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.259265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.271423 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.271709 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.275313 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.275599 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.275680 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81552) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.278824 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.279134 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.279483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.282651 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.282955 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.283407 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81552) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.286680 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81552) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.287300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.289800 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.290111 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.290444 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.290528 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.290753 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.291043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.291334 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.293543 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.293848 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.294139 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.294474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.294763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.295150 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.307126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81552) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.307419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.307696 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.307971 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.310900 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.311194 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.311472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.311750 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.315409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.316950 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.318527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.318813 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.319032 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.320704 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.321602 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81552) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.321895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.322199 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.322305 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.322517 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.322598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.322813 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.325148 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.325439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.325735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.326022 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.326342 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.334829 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81552) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.335133 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.335424 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.335716 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.336007 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.336293 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.336591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.338177 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.338474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.338762 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.339050 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.339339 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.339619 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.339919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.352029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81552) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.354969 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81552) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.355287 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.357857 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.358343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.359463 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.360990 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81552) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.361259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.361340 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.361568 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.362748 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.364240 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.364527 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.364811 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.367986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.369606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81552) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.371289 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.372513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.372773 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.372854 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.373109 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.373405 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.375661 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.376735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.377026 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.377326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.382131 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81552) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.382454 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.382745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.385233 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.385529 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.385815 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.389541 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81552) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.389838 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.390128 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.390438 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.390726 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.391005 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.391287 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.392180 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.392576 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.392872 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.393157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.393258 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81551) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:19:54.393431 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.393549 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.393708 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.393840 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.394002 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81552) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.394124 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.394429 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.394711 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.394994 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.395281 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.395567 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.395847 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.396123 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.396390 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.396658 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.396944 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81551) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:19:54.428787 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d5575939c0 pid: 81552 (default app) Sep 01 16:19:54.431394 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d5575939c0 pid: 81551 (default app) Sep 01 16:19:54.432078 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b22f7-4032-4fca-a82b-6a4ed7306718 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:19:54.432373 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b22f7-4032-4fca-a82b-6a4ed7306718 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:19:54.432654 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b22f7-4032-4fca-a82b-6a4ed7306718 None None] PATH_INFO: `/v3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:19:54.436983 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1/1] 10.222.0.18 () {50 vars in 791 bytes} [Thu Sep 1 16:19:54 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:00.918253 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a10d08f-472f-485d-97d9-0bb1c47bde27 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:00.918789 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a10d08f-472f-485d-97d9-0bb1c47bde27 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:00.918877 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a10d08f-472f-485d-97d9-0bb1c47bde27 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:00.923676 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1/2] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:00 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:20:00.929119 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1473a4-2e73-4daf-8a5c-b933805ca324 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:00.929635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1473a4-2e73-4daf-8a5c-b933805ca324 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:00.929729 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1473a4-2e73-4daf-8a5c-b933805ca324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:00.998449 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: INFO dbcounter [None req-7a1473a4-2e73-4daf-8a5c-b933805ca324 None None] Registered counter for database keystone Sep 01 16:20:01.017375 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG oslo_db.sqlalchemy.engines [None req-7a1473a4-2e73-4daf-8a5c-b933805ca324 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81552) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 16:20:01.023584 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writer thread running {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 16:20:01.202441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG passlib.handlers.bcrypt [None req-7a1473a4-2e73-4daf-8a5c-b933805ca324 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81552) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 16:20:01.202753 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG passlib.handlers.bcrypt [None req-7a1473a4-2e73-4daf-8a5c-b933805ca324 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81552) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 16:20:01.260700 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-7a1473a4-2e73-4daf-8a5c-b933805ca324 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:01.262326 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-7a1473a4-2e73-4daf-8a5c-b933805ca324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:01.390452 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 2/3] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:00 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 01 16:20:01.835954 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a10d08f-472f-485d-97d9-0bb1c47bde27 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:01.904174 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: INFO dbcounter [None req-0a10d08f-472f-485d-97d9-0bb1c47bde27 None None] Registered counter for database keystone Sep 01 16:20:01.925236 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG oslo_db.sqlalchemy.engines [None req-0a10d08f-472f-485d-97d9-0bb1c47bde27 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81551) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 16:20:01.931321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writer thread running {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 16:20:02.048616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a10d08f-472f-485d-97d9-0bb1c47bde27 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:02.052825 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a9aaf-cc70-4c60-93a7-c0cd56da4dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:02.053440 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a9aaf-cc70-4c60-93a7-c0cd56da4dca admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:02.053721 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a9aaf-cc70-4c60-93a7-c0cd56da4dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:02.053993 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a9aaf-cc70-4c60-93a7-c0cd56da4dca admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:02.061899 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441a9aaf-cc70-4c60-93a7-c0cd56da4dca admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:02.114171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: WARNING keystone.common.rbac_enforcer.enforcer [None req-441a9aaf-cc70-4c60-93a7-c0cd56da4dca admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 16:20:02.114511 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441a9aaf-cc70-4c60-93a7-c0cd56da4dca admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:02.123682 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 2/4] 10.222.0.18 () {58 vars in 1324 bytes} [Thu Sep 1 16:20:01 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:20:02.130475 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1473a4-2e73-4daf-8a5c-b933805ca324 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:02.144788 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1473a4-2e73-4daf-8a5c-b933805ca324 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:02.149410 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e195e8a-0d5d-4a87-9978-f85e3831127a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:02.150541 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e195e8a-0d5d-4a87-9978-f85e3831127a admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:02.150829 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e195e8a-0d5d-4a87-9978-f85e3831127a admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:02.151107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e195e8a-0d5d-4a87-9978-f85e3831127a admin admin] PATH_INFO: `/v3/projects/0d95c560fa484406b7dcc65daad0e99e` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:02.152669 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e195e8a-0d5d-4a87-9978-f85e3831127a admin admin] RBAC: Authorizing `identity:get_project(project_id=0d95c560fa484406b7dcc65daad0e99e)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:02.206050 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3e195e8a-0d5d-4a87-9978-f85e3831127a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 16:20:02.206430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e195e8a-0d5d-4a87-9978-f85e3831127a admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:02.210940 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 3/5] 10.222.0.18 () {56 vars in 1207 bytes} [Thu Sep 1 16:20:02 2022] GET /identity/v3/projects/0d95c560fa484406b7dcc65daad0e99e => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:20:04.045899 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43adb561-6670-4cf8-9f1e-d302ebb45026 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:04.046439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43adb561-6670-4cf8-9f1e-d302ebb45026 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:04.046518 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43adb561-6670-4cf8-9f1e-d302ebb45026 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:04.051173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 3/6] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:04.056359 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef4d93-683c-42f3-ba64-7f7eaf7bfe64 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:04.056764 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef4d93-683c-42f3-ba64-7f7eaf7bfe64 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:04.056920 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef4d93-683c-42f3-ba64-7f7eaf7bfe64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:04.076678 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-03ef4d93-683c-42f3-ba64-7f7eaf7bfe64 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:04.078202 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-03ef4d93-683c-42f3-ba64-7f7eaf7bfe64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:04.085977 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 4/7] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:04 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 01 16:20:04.351841 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43adb561-6670-4cf8-9f1e-d302ebb45026 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:04.364173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43adb561-6670-4cf8-9f1e-d302ebb45026 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:04.368340 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15303ac0-9569-4a79-baba-8e4ca0b6f08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:04.368934 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15303ac0-9569-4a79-baba-8e4ca0b6f08c admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:04.369215 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15303ac0-9569-4a79-baba-8e4ca0b6f08c admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:04.369490 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15303ac0-9569-4a79-baba-8e4ca0b6f08c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:04.377871 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15303ac0-9569-4a79-baba-8e4ca0b6f08c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:04.379682 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15303ac0-9569-4a79-baba-8e4ca0b6f08c admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:04.388167 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 4/8] 10.222.0.18 () {58 vars in 1324 bytes} [Thu Sep 1 16:20:04 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 16:20:04.393582 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef4d93-683c-42f3-ba64-7f7eaf7bfe64 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:04.402616 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef4d93-683c-42f3-ba64-7f7eaf7bfe64 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:04.407085 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a8993-8ebf-478d-be2c-88ff3c6dce87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:04.407706 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a8993-8ebf-478d-be2c-88ff3c6dce87 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:04.407992 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a8993-8ebf-478d-be2c-88ff3c6dce87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:04.408272 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a8993-8ebf-478d-be2c-88ff3c6dce87 admin admin] PATH_INFO: `/v3/users/eed887c1c93d4f24959b0f90e215c424` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:04.409574 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a8993-8ebf-478d-be2c-88ff3c6dce87 admin admin] RBAC: Authorizing `identity:get_user(user_id=eed887c1c93d4f24959b0f90e215c424)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:04.411424 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a8993-8ebf-478d-be2c-88ff3c6dce87 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:04.412930 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 5/9] 10.222.0.18 () {56 vars in 1198 bytes} [Thu Sep 1 16:20:04 2022] GET /identity/v3/users/eed887c1c93d4f24959b0f90e215c424 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:06.354939 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe524e-7702-467b-8362-fe0aba9d03e8 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:06.355513 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe524e-7702-467b-8362-fe0aba9d03e8 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:06.355513 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe524e-7702-467b-8362-fe0aba9d03e8 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:06.356813 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 5/10] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:06.361877 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac7764-75b5-442a-8efa-061e577ee033 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:06.362331 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac7764-75b5-442a-8efa-061e577ee033 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:06.362513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac7764-75b5-442a-8efa-061e577ee033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:06.380802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-fbac7764-75b5-442a-8efa-061e577ee033 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:06.381978 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-fbac7764-75b5-442a-8efa-061e577ee033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:06.383601 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ea03e-e4ea-4e37-a86c-0bfdfc571523 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:06.384009 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ea03e-e4ea-4e37-a86c-0bfdfc571523 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:06.384287 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ea03e-e4ea-4e37-a86c-0bfdfc571523 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:06.385588 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 6/11] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:20:06.390414 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486eb725-9359-4828-aa5b-b2102c567167 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:06.390745 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486eb725-9359-4828-aa5b-b2102c567167 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:06.391586 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486eb725-9359-4828-aa5b-b2102c567167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:06.433314 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG passlib.handlers.bcrypt [None req-486eb725-9359-4828-aa5b-b2102c567167 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81551) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 16:20:06.433709 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG passlib.handlers.bcrypt [None req-486eb725-9359-4828-aa5b-b2102c567167 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81551) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 16:20:06.460374 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 6/12] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:06.483295 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-486eb725-9359-4828-aa5b-b2102c567167 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:06.485396 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-486eb725-9359-4828-aa5b-b2102c567167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:06.583635 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 7/13] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:20:06.726837 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac7764-75b5-442a-8efa-061e577ee033 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:06.751128 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:06.831720 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac7764-75b5-442a-8efa-061e577ee033 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:06.853578 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486eb725-9359-4828-aa5b-b2102c567167 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:06.866103 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0d12c-7dbd-4443-9998-77e8c856c0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:06.866771 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0d12c-7dbd-4443-9998-77e8c856c0f5 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:06.867055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0d12c-7dbd-4443-9998-77e8c856c0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:06.867332 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0d12c-7dbd-4443-9998-77e8c856c0f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:06.878887 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e0d12c-7dbd-4443-9998-77e8c856c0f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:06.887822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-37e0d12c-7dbd-4443-9998-77e8c856c0f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:20:06.887822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:06.887822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:06.887822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:06.887822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:06.887822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:06.887822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:06.887822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:06.887822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:06.887822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:06.887822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:06.887822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:06.887822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:06.887822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:06.887822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:06.887822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:06.888907 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:06.888907 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:06.888907 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:06.888907 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:06.888907 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:06.888907 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:06.888907 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:06.888907 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:06.888907 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:06.888907 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:06.888907 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:06.888907 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:06.888907 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:06.888907 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:06.888907 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:06.888907 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:06.889844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:06.889844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:06.889844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:06.889844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:06.889844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:06.889844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:06.889844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:06.889844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:06.889844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:06.889844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:06.889844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:20:06.889844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:20:06.889844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 7/14] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:20:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:06.896315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0d12c-7dbd-4443-9998-77e8c856c0f5 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:06.954413 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486eb725-9359-4828-aa5b-b2102c567167 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:06.956688 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:06.978071 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:06.991950 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf393d8-7d9e-4af6-a7de-3021c194a1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:06.992805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf393d8-7d9e-4af6-a7de-3021c194a1f3 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:06.993089 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf393d8-7d9e-4af6-a7de-3021c194a1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:06.993364 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf393d8-7d9e-4af6-a7de-3021c194a1f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:06.997433 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0d12c-7dbd-4443-9998-77e8c856c0f5 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:07.030602 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4ca8b-3dc6-4c3d-a91f-cfd9cdfa13ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:07.031702 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4ca8b-3dc6-4c3d-a91f-cfd9cdfa13ad None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:07.031986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4ca8b-3dc6-4c3d-a91f-cfd9cdfa13ad None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:07.032259 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4ca8b-3dc6-4c3d-a91f-cfd9cdfa13ad None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:07.032917 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd4ca8b-3dc6-4c3d-a91f-cfd9cdfa13ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:07.033218 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd4ca8b-3dc6-4c3d-a91f-cfd9cdfa13ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:07.034635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd4ca8b-3dc6-4c3d-a91f-cfd9cdfa13ad None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:07.046029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 8/15] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:20:07.052065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4ca8b-3dc6-4c3d-a91f-cfd9cdfa13ad None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:07.061150 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf393d8-7d9e-4af6-a7de-3021c194a1f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:07.062763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf393d8-7d9e-4af6-a7de-3021c194a1f3 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:07.149351 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4ca8b-3dc6-4c3d-a91f-cfd9cdfa13ad None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:07.159511 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 8/16] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:20:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 16:20:07.165183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf393d8-7d9e-4af6-a7de-3021c194a1f3 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:07.168220 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:07.184656 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a584e-ac9f-4afe-ad42-a2b0c2f211bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:07.185319 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a584e-ac9f-4afe-ad42-a2b0c2f211bf None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:07.185602 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a584e-ac9f-4afe-ad42-a2b0c2f211bf None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:07.185877 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a584e-ac9f-4afe-ad42-a2b0c2f211bf None admin] PATH_INFO: `/v3/users/eed887c1c93d4f24959b0f90e215c424` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:07.187039 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a584e-ac9f-4afe-ad42-a2b0c2f211bf None admin] RBAC: Authorizing `identity:get_user(user_id=eed887c1c93d4f24959b0f90e215c424)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:07.188492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a584e-ac9f-4afe-ad42-a2b0c2f211bf None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:07.190029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 9/17] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:07 2022] GET /identity/v3/users/eed887c1c93d4f24959b0f90e215c424 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:20:07.195404 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a584e-ac9f-4afe-ad42-a2b0c2f211bf None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:07.209934 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:07.261885 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf393d8-7d9e-4af6-a7de-3021c194a1f3 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:07.293784 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a584e-ac9f-4afe-ad42-a2b0c2f211bf None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:07.295873 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c69d4-4ebc-45b9-a9ad-a942a25397d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:07.296716 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c69d4-4ebc-45b9-a9ad-a942a25397d7 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:07.296998 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c69d4-4ebc-45b9-a9ad-a942a25397d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:07.297267 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c69d4-4ebc-45b9-a9ad-a942a25397d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:07.315495 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c69d4-4ebc-45b9-a9ad-a942a25397d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:07.316945 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c69d4-4ebc-45b9-a9ad-a942a25397d7 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:07.328729 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3aa3d-c3cd-48aa-a7dd-3f4ab0053625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:07.329359 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3aa3d-c3cd-48aa-a7dd-3f4ab0053625 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:07.329643 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3aa3d-c3cd-48aa-a7dd-3f4ab0053625 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:07.329919 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3aa3d-c3cd-48aa-a7dd-3f4ab0053625 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:07.331079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3aa3d-c3cd-48aa-a7dd-3f4ab0053625 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:07.332514 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3aa3d-c3cd-48aa-a7dd-3f4ab0053625 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:07.334066 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 10/18] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:20:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:20:07.334933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-c90c69d4-4ebc-45b9-a9ad-a942a25397d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:20:07.334933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:07.334933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:07.334933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:07.334933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:07.334933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:07.334933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:07.334933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:07.334933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:07.334933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:07.334933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:07.334933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:07.334933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:20:07.334933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:20:07.334933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:20:07.334933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:20:07.335788 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:07.335788 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:07.335788 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:07.335788 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:07.335788 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:07.335788 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:07.335788 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:07.335788 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:07.335788 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:07.335788 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:07.335788 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:07.335788 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:07.335788 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:07.335788 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:07.335788 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:07.335788 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:07.335788 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:20:07.336579 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:20:07.336579 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:20:07.336579 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:20:07.336579 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 9/19] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:20:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:20:07.340010 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3aa3d-c3cd-48aa-a7dd-3f4ab0053625 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:07.341514 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c69d4-4ebc-45b9-a9ad-a942a25397d7 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:07.401707 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:07.424065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:07.442926 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3aa3d-c3cd-48aa-a7dd-3f4ab0053625 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:07.447707 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c69d4-4ebc-45b9-a9ad-a942a25397d7 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:07.475516 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46636cef-e94e-43c0-883e-0ba2b2b5fedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:07.476125 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46636cef-e94e-43c0-883e-0ba2b2b5fedf None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:07.476402 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46636cef-e94e-43c0-883e-0ba2b2b5fedf None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:07.476678 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46636cef-e94e-43c0-883e-0ba2b2b5fedf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:07.477397 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46636cef-e94e-43c0-883e-0ba2b2b5fedf None admin] RBAC: Adding query filter params (role.id=e8ba063905484d22993bcf04830091a3, scope.domain.id=default, user.id=eed887c1c93d4f24959b0f90e215c424) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:07.477705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46636cef-e94e-43c0-883e-0ba2b2b5fedf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:07.479172 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46636cef-e94e-43c0-883e-0ba2b2b5fedf None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:07.481744 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce737b89-d636-4003-9ca0-f64033e900bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:07.482385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce737b89-d636-4003-9ca0-f64033e900bf None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:07.482673 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce737b89-d636-4003-9ca0-f64033e900bf None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:07.482953 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce737b89-d636-4003-9ca0-f64033e900bf None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:07.483587 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce737b89-d636-4003-9ca0-f64033e900bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:07.483891 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce737b89-d636-4003-9ca0-f64033e900bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:07.485782 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce737b89-d636-4003-9ca0-f64033e900bf None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:07.486152 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 11/20] 10.222.0.18 () {56 vars in 1322 bytes} [Thu Sep 1 16:20:07 2022] GET /identity/v3/role_assignments?user.id=eed887c1c93d4f24959b0f90e215c424&scope.domain.id=default&role.id=e8ba063905484d22993bcf04830091a3 => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:20:07.505648 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 10/21] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:20:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:09.721302 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f70fc-3a9d-45f9-b4a0-d96258dfe05a None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:09.722332 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f70fc-3a9d-45f9-b4a0-d96258dfe05a None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:09.722845 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f70fc-3a9d-45f9-b4a0-d96258dfe05a None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:09.724390 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 12/22] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:09.729929 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f10dbe-2979-4158-a127-dc3d22532ad2 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:09.730849 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f10dbe-2979-4158-a127-dc3d22532ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:09.731328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f10dbe-2979-4158-a127-dc3d22532ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:09.758532 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-06f10dbe-2979-4158-a127-dc3d22532ad2 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:09.760085 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-06f10dbe-2979-4158-a127-dc3d22532ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:09.815585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7126a2-5a90-433a-a025-1a1cc0822e00 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:09.816232 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7126a2-5a90-433a-a025-1a1cc0822e00 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:09.816712 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7126a2-5a90-433a-a025-1a1cc0822e00 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:09.818258 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 13/23] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:20:09.824228 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b400f-f769-4c69-9568-6c45447ab506 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:09.824790 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b400f-f769-4c69-9568-6c45447ab506 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:09.825276 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b400f-f769-4c69-9568-6c45447ab506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:09.851063 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-8c1b400f-f769-4c69-9568-6c45447ab506 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:09.854540 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-8c1b400f-f769-4c69-9568-6c45447ab506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:09.870054 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 11/24] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:09.925919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0160ed0a-1af0-4eb5-bd95-93664e64a5e7 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:09.926638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0160ed0a-1af0-4eb5-bd95-93664e64a5e7 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:09.927145 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0160ed0a-1af0-4eb5-bd95-93664e64a5e7 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:09.928704 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 12/25] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:20:09.936177 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a43af-0c68-45fb-841c-d8520a56c50a None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:09.936732 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a43af-0c68-45fb-841c-d8520a56c50a None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:09.937208 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a43af-0c68-45fb-841c-d8520a56c50a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:09.953669 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 14/26] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:20:09.970608 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-077a43af-0c68-45fb-841c-d8520a56c50a None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:09.972142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-077a43af-0c68-45fb-841c-d8520a56c50a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:10.022708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:10.049222 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 13/27] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:20:10.305350 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd7656-a38f-41f4-a8e6-4a20a39f85f0 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:10.305729 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd7656-a38f-41f4-a8e6-4a20a39f85f0 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:10.305921 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd7656-a38f-41f4-a8e6-4a20a39f85f0 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:10.307283 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 15/28] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:10.312625 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436371d-ffbd-4338-b07e-6894ae5167f4 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:10.312919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436371d-ffbd-4338-b07e-6894ae5167f4 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:10.313256 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436371d-ffbd-4338-b07e-6894ae5167f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:10.336116 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-a436371d-ffbd-4338-b07e-6894ae5167f4 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:10.336280 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd7656-a38f-41f4-a8e6-4a20a39f85f0 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:10.337322 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a436371d-ffbd-4338-b07e-6894ae5167f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:10.459584 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 14/29] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:10.460553 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436371d-ffbd-4338-b07e-6894ae5167f4 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:10.475342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:10.532481 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd7656-a38f-41f4-a8e6-4a20a39f85f0 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:10.580867 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436371d-ffbd-4338-b07e-6894ae5167f4 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:10.594755 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:10.607287 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d670d-e346-46ab-b813-f02d58cd6877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:10.608298 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d670d-e346-46ab-b813-f02d58cd6877 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:10.610385 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d670d-e346-46ab-b813-f02d58cd6877 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:10.610886 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d670d-e346-46ab-b813-f02d58cd6877 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:10.638543 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2d670d-e346-46ab-b813-f02d58cd6877 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:10.645494 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-1b2d670d-e346-46ab-b813-f02d58cd6877 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:20:10.645494 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:10.645494 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:10.645494 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:10.645494 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:10.645494 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:10.645494 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:10.645494 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:10.645494 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:10.645494 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:10.645494 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:10.645494 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:10.645494 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:10.645494 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:10.645494 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:10.645494 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:10.646527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:10.646527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:10.646527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:10.646527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:10.646527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:10.646527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:10.646527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:10.646527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:10.646527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:10.646527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:10.646527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:10.646527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:10.646527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:10.646527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:10.646527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:10.646527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:10.647478 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:10.647478 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:10.647478 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:10.647478 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:10.647478 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:10.647478 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:10.647478 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:10.647478 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:10.647478 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:10.647478 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:10.647478 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:20:10.647478 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:20:10.650718 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cfb65d-ea58-4af2-9cc8-77ad58e89ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:10.651522 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfb65d-ea58-4af2-9cc8-77ad58e89ffc None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:10.651962 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfb65d-ea58-4af2-9cc8-77ad58e89ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:10.652375 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfb65d-ea58-4af2-9cc8-77ad58e89ffc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:10.654082 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 16/30] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:20:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:10.656603 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d670d-e346-46ab-b813-f02d58cd6877 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:10.674417 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cfb65d-ea58-4af2-9cc8-77ad58e89ffc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:10.676168 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cfb65d-ea58-4af2-9cc8-77ad58e89ffc None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:10.707362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-b7cfb65d-ea58-4af2-9cc8-77ad58e89ffc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:20:10.707362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:10.707362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:10.707362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:10.707362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:10.707362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:10.707362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:10.707362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:10.707362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:10.707362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:10.707362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:10.707362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:10.707362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:20:10.707362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:20:10.707362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:20:10.707362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:20:10.708468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:10.708468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:10.708468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:10.708468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:10.708468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:10.708468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:10.708468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:10.708468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:10.708468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:10.708468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:10.708468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:10.708468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:10.708468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:10.708468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:10.708468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:10.708468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:10.708468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:20:10.709453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:20:10.709453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:20:10.709453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:20:10.716221 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 15/31] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:20:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:10.726736 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cfb65d-ea58-4af2-9cc8-77ad58e89ffc None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:10.813200 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d670d-e346-46ab-b813-f02d58cd6877 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:10.850468 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d4df24-80d6-4a04-906f-b45753544433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:10.851301 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d4df24-80d6-4a04-906f-b45753544433 None admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:10.851735 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d4df24-80d6-4a04-906f-b45753544433 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:10.852145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d4df24-80d6-4a04-906f-b45753544433 None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:10.852950 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d4df24-80d6-4a04-906f-b45753544433 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:10.853116 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cfb65d-ea58-4af2-9cc8-77ad58e89ffc None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:10.855086 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d4df24-80d6-4a04-906f-b45753544433 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:10.858993 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:10.892200 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b3f4f2-1db3-4f13-9460-c1b58ca40778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:10.893064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3f4f2-1db3-4f13-9460-c1b58ca40778 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:10.893495 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3f4f2-1db3-4f13-9460-c1b58ca40778 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:10.893955 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3f4f2-1db3-4f13-9460-c1b58ca40778 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:10.894803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b3f4f2-1db3-4f13-9460-c1b58ca40778 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:10.895310 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b3f4f2-1db3-4f13-9460-c1b58ca40778 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:10.896899 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b3f4f2-1db3-4f13-9460-c1b58ca40778 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:10.910459 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 16/32] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:10.911472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b3f4f2-1db3-4f13-9460-c1b58ca40778 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:10.935415 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:11.016675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 17/33] 10.222.0.18 () {60 vars in 1131 bytes} [Thu Sep 1 16:20:10 2022] POST /identity/v3/roles => generated 219 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:20:11.016675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d4df24-80d6-4a04-906f-b45753544433 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:11.040552 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b3f4f2-1db3-4f13-9460-c1b58ca40778 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:11.108835 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3943b15-d4ff-4f44-815d-760e54bc3875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:11.110025 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3943b15-d4ff-4f44-815d-760e54bc3875 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:11.110339 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3943b15-d4ff-4f44-815d-760e54bc3875 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:11.110619 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3943b15-d4ff-4f44-815d-760e54bc3875 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:11.111276 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3943b15-d4ff-4f44-815d-760e54bc3875 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:11.111584 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3943b15-d4ff-4f44-815d-760e54bc3875 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:11.113205 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3943b15-d4ff-4f44-815d-760e54bc3875 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:11.127151 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 17/34] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:11.128135 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3943b15-d4ff-4f44-815d-760e54bc3875 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:11.131845 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d4df24-80d6-4a04-906f-b45753544433 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:11.169002 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e642f-5ffa-45dd-804c-9154a70d02a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:11.169674 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e642f-5ffa-45dd-804c-9154a70d02a8 None admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:11.169928 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e642f-5ffa-45dd-804c-9154a70d02a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:11.170204 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e642f-5ffa-45dd-804c-9154a70d02a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:11.170925 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e642f-5ffa-45dd-804c-9154a70d02a8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:11.173067 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e642f-5ffa-45dd-804c-9154a70d02a8 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:11.192122 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 18/35] 10.222.0.18 () {60 vars in 1131 bytes} [Thu Sep 1 16:20:11 2022] POST /identity/v3/roles => generated 225 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:20:11.193712 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da67bd2-ef2e-401e-9d11-181b58a0467f None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:11.194009 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da67bd2-ef2e-401e-9d11-181b58a0467f None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:11.194306 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da67bd2-ef2e-401e-9d11-181b58a0467f None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:11.195875 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 19/36] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:11.196429 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da67bd2-ef2e-401e-9d11-181b58a0467f None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:11.233669 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:11.256609 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3943b15-d4ff-4f44-815d-760e54bc3875 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:11.271823 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:11.295091 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb7376-c1f8-4bda-ae38-47326b4b6869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:11.295751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb7376-c1f8-4bda-ae38-47326b4b6869 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:11.296033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb7376-c1f8-4bda-ae38-47326b4b6869 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:11.296307 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb7376-c1f8-4bda-ae38-47326b4b6869 None admin] PATH_INFO: `/v3/users/eed887c1c93d4f24959b0f90e215c424` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:11.298566 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cb7376-c1f8-4bda-ae38-47326b4b6869 None admin] RBAC: Authorizing `identity:get_user(user_id=eed887c1c93d4f24959b0f90e215c424)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:11.300039 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cb7376-c1f8-4bda-ae38-47326b4b6869 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:11.301851 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 18/37] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:11 2022] GET /identity/v3/users/eed887c1c93d4f24959b0f90e215c424 => generated 247 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:11.304260 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadd69c-d0a8-435f-a3f2-2aedd1981d2d None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:11.304545 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadd69c-d0a8-435f-a3f2-2aedd1981d2d None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:11.304826 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadd69c-d0a8-435f-a3f2-2aedd1981d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:11.323780 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-4eadd69c-d0a8-435f-a3f2-2aedd1981d2d None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:11.325010 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-4eadd69c-d0a8-435f-a3f2-2aedd1981d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:11.327016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da67bd2-ef2e-401e-9d11-181b58a0467f None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:11.374712 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05def9dd-0da0-4fd6-88ab-319567dd4f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:11.375960 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05def9dd-0da0-4fd6-88ab-319567dd4f47 None admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:11.376439 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05def9dd-0da0-4fd6-88ab-319567dd4f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:11.376884 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05def9dd-0da0-4fd6-88ab-319567dd4f47 None admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:11.377906 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05def9dd-0da0-4fd6-88ab-319567dd4f47 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:11.379781 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05def9dd-0da0-4fd6-88ab-319567dd4f47 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:11.409763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 19/38] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:11.411235 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eadd69c-d0a8-435f-a3f2-2aedd1981d2d None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:11.417257 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 20/39] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Sep 1 16:20:11 2022] POST /identity/v3/projects => generated 301 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:20:11.505908 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:11.535713 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eadd69c-d0a8-435f-a3f2-2aedd1981d2d None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:11.576206 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed20231-5879-4bd6-9715-c45f12c56197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:11.576884 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed20231-5879-4bd6-9715-c45f12c56197 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:11.577169 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed20231-5879-4bd6-9715-c45f12c56197 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:11.577509 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed20231-5879-4bd6-9715-c45f12c56197 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:11.578850 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed20231-5879-4bd6-9715-c45f12c56197 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:11.580339 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed20231-5879-4bd6-9715-c45f12c56197 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:11.582378 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 20/40] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:20:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:11.588241 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05def9dd-0da0-4fd6-88ab-319567dd4f47 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:11.637011 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3779072-bcd8-4fa5-871a-f8b371d31532 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:11.637568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3779072-bcd8-4fa5-871a-f8b371d31532 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:11.638021 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3779072-bcd8-4fa5-871a-f8b371d31532 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:11.639911 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 21/41] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:11.646762 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649e8fa-df40-4536-b1ff-ac86983037a2 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:11.647052 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649e8fa-df40-4536-b1ff-ac86983037a2 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:11.647331 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649e8fa-df40-4536-b1ff-ac86983037a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:11.667726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-b649e8fa-df40-4536-b1ff-ac86983037a2 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:11.668920 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b649e8fa-df40-4536-b1ff-ac86983037a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:11.702203 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05def9dd-0da0-4fd6-88ab-319567dd4f47 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:11.744237 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 22/42] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:11.745087 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649e8fa-df40-4536-b1ff-ac86983037a2 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:11.819145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:11.829002 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:11.852663 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154da72a-0341-4147-be44-80e50af63e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:11.853424 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154da72a-0341-4147-be44-80e50af63e71 None admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:11.853716 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154da72a-0341-4147-be44-80e50af63e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:11.854003 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154da72a-0341-4147-be44-80e50af63e71 None admin] PATH_INFO: `/v3/domains/default/users/eed887c1c93d4f24959b0f90e215c424/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:11.854962 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649e8fa-df40-4536-b1ff-ac86983037a2 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:11.856110 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154da72a-0341-4147-be44-80e50af63e71 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=eed887c1c93d4f24959b0f90e215c424, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:11.857588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154da72a-0341-4147-be44-80e50af63e71 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:11.872273 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 21/43] 10.222.0.18 () {58 vars in 1361 bytes} [Thu Sep 1 16:20:11 2022] PUT /identity/v3/domains/default/users/eed887c1c93d4f24959b0f90e215c424/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:20:11.901046 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de2d21a-949b-4539-bf29-7de1352fa36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:11.901600 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de2d21a-949b-4539-bf29-7de1352fa36f None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:11.901933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de2d21a-949b-4539-bf29-7de1352fa36f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:11.902209 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de2d21a-949b-4539-bf29-7de1352fa36f None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:11.902918 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de2d21a-949b-4539-bf29-7de1352fa36f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:11.904345 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de2d21a-949b-4539-bf29-7de1352fa36f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:12.011313 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154da72a-0341-4147-be44-80e50af63e71 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:12.030161 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 23/44] 10.222.0.18 () {60 vars in 1131 bytes} [Thu Sep 1 16:20:11 2022] POST /identity/v3/roles => generated 223 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:20:12.131088 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154da72a-0341-4147-be44-80e50af63e71 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:12.167349 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe0d71-121e-42f5-baac-8cf26c95fb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:12.167933 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe0d71-121e-42f5-baac-8cf26c95fb0a None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:12.168220 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe0d71-121e-42f5-baac-8cf26c95fb0a None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:12.168500 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe0d71-121e-42f5-baac-8cf26c95fb0a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:12.169621 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbe0d71-121e-42f5-baac-8cf26c95fb0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:12.171110 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbe0d71-121e-42f5-baac-8cf26c95fb0a None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:12.172908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 22/45] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:20:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:12.178460 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de2d21a-949b-4539-bf29-7de1352fa36f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:12.283220 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de2d21a-949b-4539-bf29-7de1352fa36f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:12.315793 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62241154-6933-4142-8119-37cc9957ad91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:12.316416 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62241154-6933-4142-8119-37cc9957ad91 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:12.316705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62241154-6933-4142-8119-37cc9957ad91 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:12.318520 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62241154-6933-4142-8119-37cc9957ad91 None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:12.318520 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62241154-6933-4142-8119-37cc9957ad91 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:12.319218 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62241154-6933-4142-8119-37cc9957ad91 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:12.325009 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:12.349303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 24/46] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Sep 1 16:20:12 2022] POST /identity/v3/projects => generated 312 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:20:13.779315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0b262-7a86-453a-b710-1211d80db1ba None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:13.779798 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0b262-7a86-453a-b710-1211d80db1ba None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:13.779798 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0b262-7a86-453a-b710-1211d80db1ba None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:13.781139 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 23/47] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:13.786448 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88c68a-a965-4404-ba88-822df05d7682 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:13.786869 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88c68a-a965-4404-ba88-822df05d7682 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:13.787018 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88c68a-a965-4404-ba88-822df05d7682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:13.805259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-da88c68a-a965-4404-ba88-822df05d7682 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:13.806815 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-da88c68a-a965-4404-ba88-822df05d7682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:13.873285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 25/48] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:14.141199 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0b262-7a86-453a-b710-1211d80db1ba None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:14.176328 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:14.225730 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f8d33-3873-421f-bc98-dd163fde02f5 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:14.226027 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f8d33-3873-421f-bc98-dd163fde02f5 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:14.226341 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f8d33-3873-421f-bc98-dd163fde02f5 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:14.227707 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 26/49] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:20:14.232574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fc5ea-92c3-40d0-b58b-ff69c67e9033 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:14.232869 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fc5ea-92c3-40d0-b58b-ff69c67e9033 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:14.233153 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fc5ea-92c3-40d0-b58b-ff69c67e9033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:14.245549 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0b262-7a86-453a-b710-1211d80db1ba None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:14.282305 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395fe41-5f63-4905-8c66-55052a577de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:14.282965 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395fe41-5f63-4905-8c66-55052a577de4 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:14.283253 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395fe41-5f63-4905-8c66-55052a577de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:14.283534 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395fe41-5f63-4905-8c66-55052a577de4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:14.295360 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395fe41-5f63-4905-8c66-55052a577de4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:14.301021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-2395fe41-5f63-4905-8c66-55052a577de4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:20:14.301021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:14.301021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:14.301021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:14.301021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:14.301021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:14.301021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:14.301021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:14.301021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:14.301021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:14.301021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:14.301021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:14.301021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:14.301021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:14.301021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:14.301021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:14.301905 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:14.301905 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:14.301905 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:14.301905 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:14.301905 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:14.301905 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:14.301905 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:14.301905 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:14.301905 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:14.301905 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:14.301905 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:14.301905 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:14.301905 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:14.301905 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:14.301905 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:14.301905 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:14.302673 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:14.302673 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:14.302673 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:14.302673 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:14.302673 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:14.302673 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:14.302673 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:14.302673 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:14.302673 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:14.302673 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:14.302673 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:20:14.302673 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:20:14.302673 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 24/50] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:20:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:14.308699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395fe41-5f63-4905-8c66-55052a577de4 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:14.354925 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-209fc5ea-92c3-40d0-b58b-ff69c67e9033 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:14.356150 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-209fc5ea-92c3-40d0-b58b-ff69c67e9033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:14.406157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:14.413235 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395fe41-5f63-4905-8c66-55052a577de4 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:14.424432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 27/51] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:20:14.446752 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdce3c5-58b1-45b3-a3be-1637df1f0416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:14.447362 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdce3c5-58b1-45b3-a3be-1637df1f0416 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:14.447651 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdce3c5-58b1-45b3-a3be-1637df1f0416 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:14.447932 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdce3c5-58b1-45b3-a3be-1637df1f0416 None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:14.448598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdce3c5-58b1-45b3-a3be-1637df1f0416 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:14.448908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdce3c5-58b1-45b3-a3be-1637df1f0416 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:14.450316 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdce3c5-58b1-45b3-a3be-1637df1f0416 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:14.458149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 25/52] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:20:14.463626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fc5ea-92c3-40d0-b58b-ff69c67e9033 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:14.511705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:14.562256 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fc5ea-92c3-40d0-b58b-ff69c67e9033 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:14.594956 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd91fff-6c47-4941-a723-717b8a506e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:14.595589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd91fff-6c47-4941-a723-717b8a506e4f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:14.595876 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd91fff-6c47-4941-a723-717b8a506e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:14.596156 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd91fff-6c47-4941-a723-717b8a506e4f None admin] PATH_INFO: `/v3/users/eed887c1c93d4f24959b0f90e215c424` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:14.597288 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd91fff-6c47-4941-a723-717b8a506e4f None admin] RBAC: Authorizing `identity:get_user(user_id=eed887c1c93d4f24959b0f90e215c424)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:14.598760 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd91fff-6c47-4941-a723-717b8a506e4f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:14.600318 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 28/53] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:14 2022] GET /identity/v3/users/eed887c1c93d4f24959b0f90e215c424 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:20:14.605800 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdce3c5-58b1-45b3-a3be-1637df1f0416 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:14.693696 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd91fff-6c47-4941-a723-717b8a506e4f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:14.704744 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdce3c5-58b1-45b3-a3be-1637df1f0416 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:14.738080 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6fe768-e4d3-4a4c-8732-1ff2b4359d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:14.738746 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fe768-e4d3-4a4c-8732-1ff2b4359d35 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:14.739040 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fe768-e4d3-4a4c-8732-1ff2b4359d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:14.739318 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fe768-e4d3-4a4c-8732-1ff2b4359d35 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:14.740420 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6fe768-e4d3-4a4c-8732-1ff2b4359d35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:14.741867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6fe768-e4d3-4a4c-8732-1ff2b4359d35 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:14.743463 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 26/54] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:20:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:14.749114 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6fe768-e4d3-4a4c-8732-1ff2b4359d35 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:14.774495 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:14.792746 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd91fff-6c47-4941-a723-717b8a506e4f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:14.808636 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:14.826133 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e3465-4cad-4a27-b759-559538fbe20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:14.826795 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e3465-4cad-4a27-b759-559538fbe20b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:14.827080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e3465-4cad-4a27-b759-559538fbe20b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:14.827361 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e3465-4cad-4a27-b759-559538fbe20b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:14.829007 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6e3465-4cad-4a27-b759-559538fbe20b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:14.830486 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6e3465-4cad-4a27-b759-559538fbe20b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:14.832048 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 29/55] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:20:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:20:14.837449 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e3465-4cad-4a27-b759-559538fbe20b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:14.848555 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6fe768-e4d3-4a4c-8732-1ff2b4359d35 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:14.881059 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d3b45-33a6-467d-a0c6-ee2435b4b04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:14.881703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d3b45-33a6-467d-a0c6-ee2435b4b04c None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:14.881994 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d3b45-33a6-467d-a0c6-ee2435b4b04c None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:14.882293 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d3b45-33a6-467d-a0c6-ee2435b4b04c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:14.883019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d3b45-33a6-467d-a0c6-ee2435b4b04c None admin] RBAC: Adding query filter params (role.id=e8ba063905484d22993bcf04830091a3, scope.domain.id=default, user.id=eed887c1c93d4f24959b0f90e215c424) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:14.883334 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d3b45-33a6-467d-a0c6-ee2435b4b04c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:14.884769 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d3b45-33a6-467d-a0c6-ee2435b4b04c None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:14.890971 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 27/56] 10.222.0.18 () {56 vars in 1322 bytes} [Thu Sep 1 16:20:14 2022] GET /identity/v3/role_assignments?user.id=eed887c1c93d4f24959b0f90e215c424&scope.domain.id=default&role.id=e8ba063905484d22993bcf04830091a3 => generated 530 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:20:14.937827 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e3465-4cad-4a27-b759-559538fbe20b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:14.974160 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c761bc26-ccca-4e3a-bb7e-d9914517e802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:14.974736 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761bc26-ccca-4e3a-bb7e-d9914517e802 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:14.975026 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761bc26-ccca-4e3a-bb7e-d9914517e802 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:14.975298 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761bc26-ccca-4e3a-bb7e-d9914517e802 None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:14.975961 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c761bc26-ccca-4e3a-bb7e-d9914517e802 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:14.977429 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c761bc26-ccca-4e3a-bb7e-d9914517e802 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:15.007487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 30/57] 10.222.0.18 () {60 vars in 1140 bytes} [Thu Sep 1 16:20:14 2022] POST /identity/v3/projects => generated 298 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:20:16.869774 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57629e-8c5b-4410-a06c-257bf8d71fa4 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:16.870324 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57629e-8c5b-4410-a06c-257bf8d71fa4 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:16.870412 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57629e-8c5b-4410-a06c-257bf8d71fa4 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:16.871713 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 28/58] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:16.877622 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03d2c7-1ea3-4ebd-86cd-0b5914151a8c None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:16.878008 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03d2c7-1ea3-4ebd-86cd-0b5914151a8c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:16.878192 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03d2c7-1ea3-4ebd-86cd-0b5914151a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:16.896606 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-5d03d2c7-1ea3-4ebd-86cd-0b5914151a8c None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:16.897800 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-5d03d2c7-1ea3-4ebd-86cd-0b5914151a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:16.925986 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:16.963374 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 31/59] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:17.228542 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57629e-8c5b-4410-a06c-257bf8d71fa4 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:17.315934 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:17.328806 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57629e-8c5b-4410-a06c-257bf8d71fa4 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:17.361910 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ca211-61a3-4810-acaf-450fd67381ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:17.362601 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ca211-61a3-4810-acaf-450fd67381ad None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:17.362878 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ca211-61a3-4810-acaf-450fd67381ad None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:17.363154 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ca211-61a3-4810-acaf-450fd67381ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:17.364287 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440ca211-61a3-4810-acaf-450fd67381ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:17.365729 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440ca211-61a3-4810-acaf-450fd67381ad None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:17.367316 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 29/60] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:20:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:17.372698 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03d2c7-1ea3-4ebd-86cd-0b5914151a8c None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:17.467434 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03d2c7-1ea3-4ebd-86cd-0b5914151a8c None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:17.501380 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdce0ea-fc81-4bbd-88be-7824521184c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:17.501923 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdce0ea-fc81-4bbd-88be-7824521184c8 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:17.502210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdce0ea-fc81-4bbd-88be-7824521184c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:17.502521 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdce0ea-fc81-4bbd-88be-7824521184c8 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:17.503162 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdce0ea-fc81-4bbd-88be-7824521184c8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:17.504617 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdce0ea-fc81-4bbd-88be-7824521184c8 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:17.531253 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 32/61] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 1 16:20:17 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:20:20.443559 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d40525-b543-437d-b198-f32b41daaf6f None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:20.444518 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d40525-b543-437d-b198-f32b41daaf6f None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:20.445000 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d40525-b543-437d-b198-f32b41daaf6f None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:20.446670 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 30/62] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:20.452142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e8120-18ac-4f01-9796-d3efadca8e4a None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:20.453010 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e8120-18ac-4f01-9796-d3efadca8e4a None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:20.453494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e8120-18ac-4f01-9796-d3efadca8e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:20.483233 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-0b0e8120-18ac-4f01-9796-d3efadca8e4a None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:20.485363 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0b0e8120-18ac-4f01-9796-d3efadca8e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:20.505843 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:20.568879 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 33/63] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:20.688596 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0e2a0-6f43-482a-a377-b0fa53565942 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:20.689203 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0e2a0-6f43-482a-a377-b0fa53565942 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:20.689686 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0e2a0-6f43-482a-a377-b0fa53565942 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:20.691347 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 31/64] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:20.699687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98af144-424d-49a2-8545-ff792339a6d1 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:20.700570 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98af144-424d-49a2-8545-ff792339a6d1 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:20.701383 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98af144-424d-49a2-8545-ff792339a6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:20.733026 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-e98af144-424d-49a2-8545-ff792339a6d1 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:20.735133 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-e98af144-424d-49a2-8545-ff792339a6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:20.820935 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 34/65] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:20:20.830328 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0da617-ee13-47a0-b8e8-706464013185 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:20.830876 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0da617-ee13-47a0-b8e8-706464013185 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:20.831358 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0da617-ee13-47a0-b8e8-706464013185 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:20.833009 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 32/66] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:20.838282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6bf25-4f04-4938-a7f1-08e3b57cd3fe None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:20.838812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6bf25-4f04-4938-a7f1-08e3b57cd3fe None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:20.839344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6bf25-4f04-4938-a7f1-08e3b57cd3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:20.861128 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0da617-ee13-47a0-b8e8-706464013185 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:20.871547 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-0bd6bf25-4f04-4938-a7f1-08e3b57cd3fe None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:20.875769 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0bd6bf25-4f04-4938-a7f1-08e3b57cd3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:21.002315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0da617-ee13-47a0-b8e8-706464013185 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:21.028435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 35/67] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:21.052042 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce14872-03eb-448e-a42e-eed9e34856a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:21.052506 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41e106-ccc1-4bc8-952d-979a063b51be None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:21.052679 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce14872-03eb-448e-a42e-eed9e34856a2 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:21.053108 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41e106-ccc1-4bc8-952d-979a063b51be None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:21.053210 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce14872-03eb-448e-a42e-eed9e34856a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:21.053398 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce14872-03eb-448e-a42e-eed9e34856a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:21.053869 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41e106-ccc1-4bc8-952d-979a063b51be None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:21.059856 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 36/68] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:21 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:21.065441 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ba896-1cd8-4092-83ff-e1aea04fbdb0 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:21.065991 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ba896-1cd8-4092-83ff-e1aea04fbdb0 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:21.066519 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ba896-1cd8-4092-83ff-e1aea04fbdb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:21.067707 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce14872-03eb-448e-a42e-eed9e34856a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:21.072763 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-9ce14872-03eb-448e-a42e-eed9e34856a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:20:21.072763 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:21.072763 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:21.072763 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:21.072763 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:21.072763 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:21.072763 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:21.072763 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:21.072763 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:21.072763 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:21.072763 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:21.072763 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:21.072763 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:21.072763 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:21.072763 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:21.072763 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:21.073575 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:21.073575 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:21.073575 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:21.073575 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:21.073575 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:21.073575 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:21.073575 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:21.073575 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:21.073575 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:21.073575 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:21.073575 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:21.073575 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:21.073575 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:21.073575 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:21.073575 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:21.073575 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:21.074606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:21.074606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:21.074606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:21.074606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:21.074606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:21.074606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:21.074606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:21.074606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:21.074606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:21.074606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:21.074606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:20:21.074606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:20:21.074606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 33/69] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:20:21.083197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce14872-03eb-448e-a42e-eed9e34856a2 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:21.092252 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-523ba896-1cd8-4092-83ff-e1aea04fbdb0 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:21.094195 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-523ba896-1cd8-4092-83ff-e1aea04fbdb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:21.102600 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:21.114833 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:21.172854 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 37/70] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:20:21.180593 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ba896-1cd8-4092-83ff-e1aea04fbdb0 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:21.221470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce14872-03eb-448e-a42e-eed9e34856a2 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:21.256322 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf5b88-0716-43a6-a75b-d9b58f7fbfb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:21.257475 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf5b88-0716-43a6-a75b-d9b58f7fbfb4 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:21.257475 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf5b88-0716-43a6-a75b-d9b58f7fbfb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:21.257475 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf5b88-0716-43a6-a75b-d9b58f7fbfb4 None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:21.258029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbf5b88-0716-43a6-a75b-d9b58f7fbfb4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:21.258358 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbf5b88-0716-43a6-a75b-d9b58f7fbfb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:21.259726 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbf5b88-0716-43a6-a75b-d9b58f7fbfb4 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:21.268078 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 34/71] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 1 16:20:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:21.269240 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d089f-2c86-491a-b3d6-d25b995c9ba2 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:21.269528 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d089f-2c86-491a-b3d6-d25b995c9ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:21.269909 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d089f-2c86-491a-b3d6-d25b995c9ba2 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:21.275462 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 35/72] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:21.276913 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d089f-2c86-491a-b3d6-d25b995c9ba2 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:21.292173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ba896-1cd8-4092-83ff-e1aea04fbdb0 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:21.329879 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f91387-abe4-4101-a431-74f7ad9b7445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:21.331083 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f91387-abe4-4101-a431-74f7ad9b7445 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:21.331555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f91387-abe4-4101-a431-74f7ad9b7445 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:21.332040 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f91387-abe4-4101-a431-74f7ad9b7445 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:21.345465 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f91387-abe4-4101-a431-74f7ad9b7445 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:21.353615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-13f91387-abe4-4101-a431-74f7ad9b7445 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:20:21.353615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:21.353615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:21.353615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:21.353615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:21.353615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:21.353615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:21.353615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:21.353615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:21.353615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:21.353615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:21.353615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:21.353615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:21.353615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:21.353615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:21.353615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:21.354512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:21.354512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:21.354512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:21.354512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:21.354512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:21.354512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:21.354512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:21.354512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:21.354512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:21.354512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:21.354512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:21.354512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:21.354512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:21.354512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:21.354512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:21.354512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:21.355224 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:21.355224 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:21.355224 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:21.355224 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:21.355224 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:21.355224 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:21.355224 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:21.355224 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:21.355224 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:21.355224 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:21.355224 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:20:21.355224 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:20:21.355224 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 38/73] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:20:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:21.356409 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b965fd-ff00-480e-ba2a-f0de3031ce27 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:21.356409 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b965fd-ff00-480e-ba2a-f0de3031ce27 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:21.356553 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b965fd-ff00-480e-ba2a-f0de3031ce27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:21.365572 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:21.370346 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:21.380776 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-e4b965fd-ff00-480e-ba2a-f0de3031ce27 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:21.381978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-e4b965fd-ff00-480e-ba2a-f0de3031ce27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:21.398197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d089f-2c86-491a-b3d6-d25b995c9ba2 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:21.436170 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e237fe-1c93-437c-8460-3e13262ed530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:21.436812 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e237fe-1c93-437c-8460-3e13262ed530 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:21.437128 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e237fe-1c93-437c-8460-3e13262ed530 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:21.437409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e237fe-1c93-437c-8460-3e13262ed530 None admin] PATH_INFO: `/v3/users/6d995078a79f4035bcbc04685a0579e2` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:21.448895 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 39/74] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:21.449749 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b965fd-ff00-480e-ba2a-f0de3031ce27 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:21.451655 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e237fe-1c93-437c-8460-3e13262ed530 None admin] RBAC: Authorizing `identity:get_user(user_id=6d995078a79f4035bcbc04685a0579e2)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:21.454182 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e237fe-1c93-437c-8460-3e13262ed530 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:21.455863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 36/75] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:21 2022] GET /identity/v3/users/6d995078a79f4035bcbc04685a0579e2 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:21.456755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e237fe-1c93-437c-8460-3e13262ed530 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:21.557234 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b965fd-ff00-480e-ba2a-f0de3031ce27 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:21.560397 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e237fe-1c93-437c-8460-3e13262ed530 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:21.588366 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:21.594159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b19d85-3e6e-4ae2-b4a8-0e6788a5b585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:21.594821 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b19d85-3e6e-4ae2-b4a8-0e6788a5b585 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:21.595107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b19d85-3e6e-4ae2-b4a8-0e6788a5b585 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:21.595286 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1185766-676f-404f-8df7-94619263b96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:21.595413 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b19d85-3e6e-4ae2-b4a8-0e6788a5b585 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:21.595875 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1185766-676f-404f-8df7-94619263b96b None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:21.596164 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1185766-676f-404f-8df7-94619263b96b None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:21.596441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1185766-676f-404f-8df7-94619263b96b None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:21.597113 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1185766-676f-404f-8df7-94619263b96b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:21.597421 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1185766-676f-404f-8df7-94619263b96b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:21.598846 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1185766-676f-404f-8df7-94619263b96b None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:21.601431 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:21.607930 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b19d85-3e6e-4ae2-b4a8-0e6788a5b585 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:21.608380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 37/76] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:21.609895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1185766-676f-404f-8df7-94619263b96b None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:21.617712 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-75b19d85-3e6e-4ae2-b4a8-0e6788a5b585 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:20:21.617712 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:21.617712 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:21.617712 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:21.617712 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:21.617712 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:21.617712 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:21.617712 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:21.617712 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:21.617712 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:21.617712 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:21.617712 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:21.617712 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:21.617712 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:21.617712 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:21.617712 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:21.618557 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:21.618557 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:21.618557 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:21.618557 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:21.618557 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:21.618557 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:21.618557 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:21.618557 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:21.618557 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:21.618557 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:21.618557 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:21.618557 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:21.618557 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:21.618557 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:21.618557 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:21.618557 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:21.619308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:21.619308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:21.619308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:21.619308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:21.619308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:21.619308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:21.619308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:21.619308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:21.619308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:21.619308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:21.619308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:20:21.619308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:20:21.619308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 40/77] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 1 16:20:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:21.620948 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b19d85-3e6e-4ae2-b4a8-0e6788a5b585 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:21.719140 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1185766-676f-404f-8df7-94619263b96b None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:21.724488 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b19d85-3e6e-4ae2-b4a8-0e6788a5b585 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:21.752845 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a08186-79e6-4a88-82d2-d2c82bd247cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:21.753478 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a08186-79e6-4a88-82d2-d2c82bd247cd None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:21.753761 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a08186-79e6-4a88-82d2-d2c82bd247cd None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:21.754037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a08186-79e6-4a88-82d2-d2c82bd247cd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:21.755170 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a08186-79e6-4a88-82d2-d2c82bd247cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:21.756624 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a08186-79e6-4a88-82d2-d2c82bd247cd None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:21.758272 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 38/78] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:20:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:21.758673 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ef6db-76bc-49a2-bc05-e6edb97b2196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:21.758991 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a08186-79e6-4a88-82d2-d2c82bd247cd None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:21.759368 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ef6db-76bc-49a2-bc05-e6edb97b2196 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:21.759677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ef6db-76bc-49a2-bc05-e6edb97b2196 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:21.759941 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ef6db-76bc-49a2-bc05-e6edb97b2196 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:21.818682 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:21.824697 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:21.828306 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ef6db-76bc-49a2-bc05-e6edb97b2196 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:21.829780 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ef6db-76bc-49a2-bc05-e6edb97b2196 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:21.860105 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a08186-79e6-4a88-82d2-d2c82bd247cd None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:21.892820 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b0eded-54ce-4957-82f0-40f344e3859b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:21.893444 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0eded-54ce-4957-82f0-40f344e3859b None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:21.893730 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0eded-54ce-4957-82f0-40f344e3859b None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:21.894008 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0eded-54ce-4957-82f0-40f344e3859b None admin] PATH_INFO: `/v3/users/eed887c1c93d4f24959b0f90e215c424` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:21.895152 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b0eded-54ce-4957-82f0-40f344e3859b None admin] RBAC: Authorizing `identity:get_user(user_id=eed887c1c93d4f24959b0f90e215c424)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:21.896592 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b0eded-54ce-4957-82f0-40f344e3859b None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:21.898164 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 39/79] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:21 2022] GET /identity/v3/users/eed887c1c93d4f24959b0f90e215c424 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:21.898947 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b0eded-54ce-4957-82f0-40f344e3859b None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:21.930660 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 41/80] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:20:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:20:21.930660 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ef6db-76bc-49a2-bc05-e6edb97b2196 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:21.998038 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b0eded-54ce-4957-82f0-40f344e3859b None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:22.015356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:22.024621 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ef6db-76bc-49a2-bc05-e6edb97b2196 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:22.030536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bccd5ff-2e49-469e-b3ac-2e0952fc0548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:22.031713 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bccd5ff-2e49-469e-b3ac-2e0952fc0548 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:22.032003 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bccd5ff-2e49-469e-b3ac-2e0952fc0548 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:22.032282 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bccd5ff-2e49-469e-b3ac-2e0952fc0548 None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:22.033304 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bccd5ff-2e49-469e-b3ac-2e0952fc0548 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:22.033304 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bccd5ff-2e49-469e-b3ac-2e0952fc0548 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:22.034663 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bccd5ff-2e49-469e-b3ac-2e0952fc0548 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:22.035996 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:22.044247 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 40/81] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:20:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:22.045031 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bccd5ff-2e49-469e-b3ac-2e0952fc0548 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:22.063049 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af79447-e418-467a-b57c-156831e651dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:22.063765 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af79447-e418-467a-b57c-156831e651dd None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:22.064118 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af79447-e418-467a-b57c-156831e651dd None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:22.064485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af79447-e418-467a-b57c-156831e651dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:22.074631 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af79447-e418-467a-b57c-156831e651dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:22.079919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-3af79447-e418-467a-b57c-156831e651dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:20:22.079919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:22.079919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:22.079919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:22.079919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:22.079919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:22.079919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:22.079919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:22.079919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:22.079919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:22.079919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:22.079919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:22.079919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:22.079919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:22.079919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:22.079919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:22.080726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:22.080726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:22.080726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:22.080726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:22.080726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:22.080726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:22.080726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:22.080726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:22.080726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:22.080726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:22.080726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:22.080726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:22.080726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:22.080726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:22.080726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:22.080726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:22.081610 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:22.081610 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:22.081610 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:22.081610 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:22.081610 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:22.081610 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:22.081610 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:22.081610 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:22.081610 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:22.081610 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:22.081610 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:20:22.081610 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:20:22.081610 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 42/82] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:22.082470 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af79447-e418-467a-b57c-156831e651dd None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:22.146936 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bccd5ff-2e49-469e-b3ac-2e0952fc0548 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:22.181106 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af79447-e418-467a-b57c-156831e651dd None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:22.181433 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7b57fa-b4a3-43e9-a174-bc3d1e19babb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:22.182052 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b57fa-b4a3-43e9-a174-bc3d1e19babb None admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:22.182375 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b57fa-b4a3-43e9-a174-bc3d1e19babb None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:22.182659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b57fa-b4a3-43e9-a174-bc3d1e19babb None admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:22.183315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7b57fa-b4a3-43e9-a174-bc3d1e19babb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:22.184770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7b57fa-b4a3-43e9-a174-bc3d1e19babb None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:22.211342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 41/83] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 1 16:20:22 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:20:22.212177 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7b57fa-b4a3-43e9-a174-bc3d1e19babb None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:22.222894 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1586f20b-9681-4f45-9adc-38a80815b9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:22.223548 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586f20b-9681-4f45-9adc-38a80815b9f1 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:22.223833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586f20b-9681-4f45-9adc-38a80815b9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:22.224110 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586f20b-9681-4f45-9adc-38a80815b9f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:22.251404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:22.292101 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:22.306072 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1586f20b-9681-4f45-9adc-38a80815b9f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:22.307894 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1586f20b-9681-4f45-9adc-38a80815b9f1 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:22.354395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7b57fa-b4a3-43e9-a174-bc3d1e19babb None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:22.402985 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbddeac-c25d-4152-8f38-6b1eb597ff42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:22.403645 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbddeac-c25d-4152-8f38-6b1eb597ff42 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:22.404045 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbddeac-c25d-4152-8f38-6b1eb597ff42 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:22.404331 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbddeac-c25d-4152-8f38-6b1eb597ff42 None admin] PATH_INFO: `/v3/projects/a3f472f7ed974397b7020b42b4f22bd3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:22.405509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbddeac-c25d-4152-8f38-6b1eb597ff42 None admin] RBAC: Authorizing `identity:get_project(project_id=a3f472f7ed974397b7020b42b4f22bd3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:22.407129 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbddeac-c25d-4152-8f38-6b1eb597ff42 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:22.410108 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 42/84] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:22 2022] GET /identity/v3/projects/a3f472f7ed974397b7020b42b4f22bd3 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:22.410108 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbddeac-c25d-4152-8f38-6b1eb597ff42 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:22.427727 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 43/85] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:20:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:20:22.428597 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1586f20b-9681-4f45-9adc-38a80815b9f1 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:22.494882 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:22.538855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbddeac-c25d-4152-8f38-6b1eb597ff42 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:22.563395 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1586f20b-9681-4f45-9adc-38a80815b9f1 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:22.563886 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:22.586022 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3c8cb-ebd5-4c2d-99ff-23e83dd80403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:22.587331 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3c8cb-ebd5-4c2d-99ff-23e83dd80403 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:22.587763 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3c8cb-ebd5-4c2d-99ff-23e83dd80403 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:22.588453 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3c8cb-ebd5-4c2d-99ff-23e83dd80403 None admin] PATH_INFO: `/v3/users/6d995078a79f4035bcbc04685a0579e2` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:22.590475 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3c8cb-ebd5-4c2d-99ff-23e83dd80403 None admin] RBAC: Authorizing `identity:get_user(user_id=6d995078a79f4035bcbc04685a0579e2)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:22.592737 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3c8cb-ebd5-4c2d-99ff-23e83dd80403 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:22.596039 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 43/86] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:22 2022] GET /identity/v3/users/6d995078a79f4035bcbc04685a0579e2 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:22.597278 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3c8cb-ebd5-4c2d-99ff-23e83dd80403 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:22.601703 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f81ad4-a191-4819-ba7f-cdd57ee76deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:22.601703 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f81ad4-a191-4819-ba7f-cdd57ee76deb None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:22.601933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f81ad4-a191-4819-ba7f-cdd57ee76deb None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:22.602195 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f81ad4-a191-4819-ba7f-cdd57ee76deb None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:22.603018 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f81ad4-a191-4819-ba7f-cdd57ee76deb None admin] RBAC: Adding query filter params (name=member) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:22.603338 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f81ad4-a191-4819-ba7f-cdd57ee76deb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:22.604836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f81ad4-a191-4819-ba7f-cdd57ee76deb None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:22.614443 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 44/87] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 1 16:20:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:22.615313 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f81ad4-a191-4819-ba7f-cdd57ee76deb None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:22.723867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3c8cb-ebd5-4c2d-99ff-23e83dd80403 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:22.745839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f81ad4-a191-4819-ba7f-cdd57ee76deb None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:22.766586 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:22.770001 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18253335-8d38-41c8-a1f6-42f8b07638d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:22.770654 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18253335-8d38-41c8-a1f6-42f8b07638d3 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:22.770944 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18253335-8d38-41c8-a1f6-42f8b07638d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:22.771226 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18253335-8d38-41c8-a1f6-42f8b07638d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:22.771967 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18253335-8d38-41c8-a1f6-42f8b07638d3 None admin] RBAC: Adding query filter params (role.id=45545334e1fc473c9d85fc91a9b0a35c, scope.project.id=a3f472f7ed974397b7020b42b4f22bd3, user.id=6d995078a79f4035bcbc04685a0579e2) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:22.772283 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18253335-8d38-41c8-a1f6-42f8b07638d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:22.773761 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18253335-8d38-41c8-a1f6-42f8b07638d3 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:22.786024 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 44/88] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:20:22 2022] GET /identity/v3/role_assignments?user.id=6d995078a79f4035bcbc04685a0579e2&scope.project.id=a3f472f7ed974397b7020b42b4f22bd3&role.id=45545334e1fc473c9d85fc91a9b0a35c => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:22.787953 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18253335-8d38-41c8-a1f6-42f8b07638d3 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:22.795492 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb81dc-1c66-4e4b-a406-dd197c6f4136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:22.796168 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb81dc-1c66-4e4b-a406-dd197c6f4136 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:22.797071 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb81dc-1c66-4e4b-a406-dd197c6f4136 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:22.797362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb81dc-1c66-4e4b-a406-dd197c6f4136 None admin] PATH_INFO: `/v3/projects/a3f472f7ed974397b7020b42b4f22bd3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:22.798744 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb81dc-1c66-4e4b-a406-dd197c6f4136 None admin] RBAC: Authorizing `identity:get_project(project_id=a3f472f7ed974397b7020b42b4f22bd3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:22.800288 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb81dc-1c66-4e4b-a406-dd197c6f4136 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:22.801990 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 45/89] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:22 2022] GET /identity/v3/projects/a3f472f7ed974397b7020b42b4f22bd3 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:22.807668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb81dc-1c66-4e4b-a406-dd197c6f4136 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:22.863807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:22.927443 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18253335-8d38-41c8-a1f6-42f8b07638d3 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:22.970668 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7435a-1eb0-4699-a010-f7d58f69696c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:22.972082 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7435a-1eb0-4699-a010-f7d58f69696c None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:22.972831 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7435a-1eb0-4699-a010-f7d58f69696c None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:22.973295 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7435a-1eb0-4699-a010-f7d58f69696c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:23.044159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb81dc-1c66-4e4b-a406-dd197c6f4136 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:23.050351 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd7435a-1eb0-4699-a010-f7d58f69696c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:23.051870 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd7435a-1eb0-4699-a010-f7d58f69696c None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:23.106097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:23.119398 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017be81-4cbe-468b-8f2d-0c8099cb32da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:23.119980 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017be81-4cbe-468b-8f2d-0c8099cb32da None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:23.120265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017be81-4cbe-468b-8f2d-0c8099cb32da None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:23.120555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017be81-4cbe-468b-8f2d-0c8099cb32da None admin] PATH_INFO: `/v3/users/6d995078a79f4035bcbc04685a0579e2` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:23.121765 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017be81-4cbe-468b-8f2d-0c8099cb32da None admin] RBAC: Authorizing `identity:get_user(user_id=6d995078a79f4035bcbc04685a0579e2)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:23.123815 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017be81-4cbe-468b-8f2d-0c8099cb32da None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:23.125697 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 46/90] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:22 2022] GET /identity/v3/users/6d995078a79f4035bcbc04685a0579e2 => generated 275 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:23.126487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017be81-4cbe-468b-8f2d-0c8099cb32da None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:23.192989 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:23.219063 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 45/91] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:20:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:20:23.219923 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7435a-1eb0-4699-a010-f7d58f69696c None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:23.282117 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017be81-4cbe-468b-8f2d-0c8099cb32da None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:23.338651 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2baf885-11b3-4d6c-abe0-9772f78d222c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:23.339347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2baf885-11b3-4d6c-abe0-9772f78d222c None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:23.339642 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2baf885-11b3-4d6c-abe0-9772f78d222c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:23.339922 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2baf885-11b3-4d6c-abe0-9772f78d222c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:23.340664 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2baf885-11b3-4d6c-abe0-9772f78d222c None admin] RBAC: Adding query filter params (role.id=e8ba063905484d22993bcf04830091a3, scope.project.id=a3f472f7ed974397b7020b42b4f22bd3, user.id=eed887c1c93d4f24959b0f90e215c424) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:23.340979 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2baf885-11b3-4d6c-abe0-9772f78d222c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:23.343455 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2baf885-11b3-4d6c-abe0-9772f78d222c None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:23.360203 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 47/92] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:20:23 2022] GET /identity/v3/role_assignments?user.id=eed887c1c93d4f24959b0f90e215c424&scope.project.id=a3f472f7ed974397b7020b42b4f22bd3&role.id=e8ba063905484d22993bcf04830091a3 => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:23.361082 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2baf885-11b3-4d6c-abe0-9772f78d222c None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:23.404394 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7435a-1eb0-4699-a010-f7d58f69696c None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:23.494550 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad3aab-3b9b-4513-98fa-00ea1d19f1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:23.495544 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad3aab-3b9b-4513-98fa-00ea1d19f1f5 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:23.496054 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad3aab-3b9b-4513-98fa-00ea1d19f1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:23.496528 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad3aab-3b9b-4513-98fa-00ea1d19f1f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:23.506084 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2baf885-11b3-4d6c-abe0-9772f78d222c None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:23.522046 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:23.549706 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:23.550264 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c8936-d36b-443a-8909-8aa2ef9d7368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:23.550264 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c8936-d36b-443a-8909-8aa2ef9d7368 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:23.550264 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c8936-d36b-443a-8909-8aa2ef9d7368 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:23.550264 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c8936-d36b-443a-8909-8aa2ef9d7368 None admin] PATH_INFO: `/v3/projects/a3f472f7ed974397b7020b42b4f22bd3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:23.551368 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95c8936-d36b-443a-8909-8aa2ef9d7368 None admin] RBAC: Authorizing `identity:get_project(project_id=a3f472f7ed974397b7020b42b4f22bd3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:23.555475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95c8936-d36b-443a-8909-8aa2ef9d7368 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:23.555475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 48/93] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:23 2022] GET /identity/v3/projects/a3f472f7ed974397b7020b42b4f22bd3 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:23.563474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c8936-d36b-443a-8909-8aa2ef9d7368 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:23.670536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad3aab-3b9b-4513-98fa-00ea1d19f1f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:23.672799 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad3aab-3b9b-4513-98fa-00ea1d19f1f5 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:23.706789 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c8936-d36b-443a-8909-8aa2ef9d7368 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:23.742879 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd7f5a-69e0-473f-a587-53e36eaf89e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:23.743660 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd7f5a-69e0-473f-a587-53e36eaf89e4 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:23.744053 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd7f5a-69e0-473f-a587-53e36eaf89e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:23.744451 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd7f5a-69e0-473f-a587-53e36eaf89e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:23.745299 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd7f5a-69e0-473f-a587-53e36eaf89e4 None admin] RBAC: Adding query filter params (role.id=f2a6e936aa294323ba7087bf377276b2, scope.project.id=a3f472f7ed974397b7020b42b4f22bd3, user.id=6d995078a79f4035bcbc04685a0579e2) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:23.745707 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd7f5a-69e0-473f-a587-53e36eaf89e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:23.747293 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd7f5a-69e0-473f-a587-53e36eaf89e4 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:23.753833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 49/94] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:20:23 2022] GET /identity/v3/role_assignments?user.id=6d995078a79f4035bcbc04685a0579e2&scope.project.id=a3f472f7ed974397b7020b42b4f22bd3&role.id=f2a6e936aa294323ba7087bf377276b2 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:23.872651 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 46/95] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:20:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 653 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:20:23.879390 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd7f5a-69e0-473f-a587-53e36eaf89e4 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:23.954127 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:24.014488 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd7f5a-69e0-473f-a587-53e36eaf89e4 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:24.050758 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96773e8-4873-4014-b0a8-818a0e85c661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:24.051419 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96773e8-4873-4014-b0a8-818a0e85c661 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:24.051708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96773e8-4873-4014-b0a8-818a0e85c661 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:24.051989 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96773e8-4873-4014-b0a8-818a0e85c661 None admin] PATH_INFO: `/v3/projects/1cf23dd85509483f9d433c73e9329b89` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:24.053148 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96773e8-4873-4014-b0a8-818a0e85c661 None admin] RBAC: Authorizing `identity:get_project(project_id=1cf23dd85509483f9d433c73e9329b89)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:24.054647 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96773e8-4873-4014-b0a8-818a0e85c661 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:24.056310 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 50/96] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:23 2022] GET /identity/v3/projects/1cf23dd85509483f9d433c73e9329b89 => generated 312 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:24.062218 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad3aab-3b9b-4513-98fa-00ea1d19f1f5 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:24.115794 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:24.176350 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad3aab-3b9b-4513-98fa-00ea1d19f1f5 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:24.221085 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447dc29b-37a2-40ef-aeaa-49ca5c143368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:24.221626 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447dc29b-37a2-40ef-aeaa-49ca5c143368 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:24.221914 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447dc29b-37a2-40ef-aeaa-49ca5c143368 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:24.222195 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447dc29b-37a2-40ef-aeaa-49ca5c143368 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:24.223030 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447dc29b-37a2-40ef-aeaa-49ca5c143368 None admin] RBAC: Adding query filter params (role.id=45545334e1fc473c9d85fc91a9b0a35c, scope.project.id=1cf23dd85509483f9d433c73e9329b89, user.id=6d995078a79f4035bcbc04685a0579e2) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:24.223350 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447dc29b-37a2-40ef-aeaa-49ca5c143368 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:24.224885 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447dc29b-37a2-40ef-aeaa-49ca5c143368 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:24.230872 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 47/97] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:20:24 2022] GET /identity/v3/role_assignments?user.id=6d995078a79f4035bcbc04685a0579e2&scope.project.id=1cf23dd85509483f9d433c73e9329b89&role.id=45545334e1fc473c9d85fc91a9b0a35c => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:24.639130 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29a4fc-5131-44c4-bda6-5cead7d0fa02 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:24.639636 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29a4fc-5131-44c4-bda6-5cead7d0fa02 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:24.639737 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29a4fc-5131-44c4-bda6-5cead7d0fa02 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:24.641198 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 51/98] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:24.649496 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbc915-eb05-429b-bf54-f2f4abc3a8fe None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:24.649890 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbc915-eb05-429b-bf54-f2f4abc3a8fe None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:24.650071 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbc915-eb05-429b-bf54-f2f4abc3a8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:24.669260 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-7dcbc915-eb05-429b-bf54-f2f4abc3a8fe None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:24.670823 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-7dcbc915-eb05-429b-bf54-f2f4abc3a8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:24.770000 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 48/99] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:20:24.914158 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73738675-d1e0-453e-98e6-f251817b4ed3 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:24.914489 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73738675-d1e0-453e-98e6-f251817b4ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:24.914684 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73738675-d1e0-453e-98e6-f251817b4ed3 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:24.916038 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 52/100] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:24.921357 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64fb73b-cfb7-4f80-8c34-3ec7d57591b3 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:24.924801 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64fb73b-cfb7-4f80-8c34-3ec7d57591b3 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:24.927485 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64fb73b-cfb7-4f80-8c34-3ec7d57591b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:24.955270 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-a64fb73b-cfb7-4f80-8c34-3ec7d57591b3 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:24.956466 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-a64fb73b-cfb7-4f80-8c34-3ec7d57591b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:25.010468 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:25.040293 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 49/101] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:25.328559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73738675-d1e0-453e-98e6-f251817b4ed3 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:25.450433 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64fb73b-cfb7-4f80-8c34-3ec7d57591b3 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:25.480962 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73738675-d1e0-453e-98e6-f251817b4ed3 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:25.507308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:25.534149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4d21b-3f51-40f8-9c62-4bd2e88a9509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:25.534831 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4d21b-3f51-40f8-9c62-4bd2e88a9509 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:25.535115 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4d21b-3f51-40f8-9c62-4bd2e88a9509 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:25.535395 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4d21b-3f51-40f8-9c62-4bd2e88a9509 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:25.546708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4d21b-3f51-40f8-9c62-4bd2e88a9509 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:25.552246 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-7fb4d21b-3f51-40f8-9c62-4bd2e88a9509 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:20:25.552246 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:25.552246 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:25.552246 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:25.552246 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:25.552246 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:25.552246 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:25.552246 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:25.552246 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:25.552246 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:25.552246 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:25.552246 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:25.552246 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:25.552246 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:25.552246 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:25.552246 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:25.553136 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:25.553136 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:25.553136 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:25.553136 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:25.553136 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:25.553136 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:25.553136 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:25.553136 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:25.553136 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:25.553136 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:25.553136 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:25.553136 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:25.553136 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:25.553136 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:25.553136 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:25.553136 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:25.553859 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:25.553859 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:25.553859 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:25.553859 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:25.553859 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:25.553859 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:25.553859 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:25.553859 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:25.553859 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:25.553859 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:25.553859 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:20:25.553859 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:20:25.553859 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 53/102] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:25.554505 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1d3d5-cfb5-4858-8901-90bedcaf7217 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:25.555274 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1d3d5-cfb5-4858-8901-90bedcaf7217 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:25.555274 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1d3d5-cfb5-4858-8901-90bedcaf7217 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:25.556352 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 54/103] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:25.561089 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f1d3d5-cfb5-4858-8901-90bedcaf7217 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:25.644776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64fb73b-cfb7-4f80-8c34-3ec7d57591b3 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:25.705772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193599d3-6c88-407f-be7e-966f5f080d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:25.706426 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193599d3-6c88-407f-be7e-966f5f080d60 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:25.706717 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193599d3-6c88-407f-be7e-966f5f080d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:25.706996 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193599d3-6c88-407f-be7e-966f5f080d60 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:25.708900 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193599d3-6c88-407f-be7e-966f5f080d60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:25.710384 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193599d3-6c88-407f-be7e-966f5f080d60 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:25.711959 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 50/104] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:20:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:20:25.715440 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e38d4f-a2dc-4ad0-bac8-7b779f5e2595 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:25.715727 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e38d4f-a2dc-4ad0-bac8-7b779f5e2595 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:25.715999 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e38d4f-a2dc-4ad0-bac8-7b779f5e2595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:25.744333 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f1d3d5-cfb5-4858-8901-90bedcaf7217 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:25.755416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-29e38d4f-a2dc-4ad0-bac8-7b779f5e2595 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:25.756811 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-29e38d4f-a2dc-4ad0-bac8-7b779f5e2595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:25.802676 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a741e-ab72-4edc-b73e-3e15cf8f6cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:25.803561 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a741e-ab72-4edc-b73e-3e15cf8f6cbd None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:25.803988 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a741e-ab72-4edc-b73e-3e15cf8f6cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:25.804406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a741e-ab72-4edc-b73e-3e15cf8f6cbd None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:25.805213 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a741e-ab72-4edc-b73e-3e15cf8f6cbd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:25.805674 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a741e-ab72-4edc-b73e-3e15cf8f6cbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:25.807218 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a741e-ab72-4edc-b73e-3e15cf8f6cbd None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:25.825419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:25.827063 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 55/105] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 1 16:20:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:25.828667 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a741e-ab72-4edc-b73e-3e15cf8f6cbd None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:25.872980 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 51/106] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:25.875085 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e38d4f-a2dc-4ad0-bac8-7b779f5e2595 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:25.893999 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:26.014928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a741e-ab72-4edc-b73e-3e15cf8f6cbd None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:26.072469 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e38d4f-a2dc-4ad0-bac8-7b779f5e2595 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:26.084359 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7c9e2-a1c1-4a16-808c-8a8ffd618659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:26.084997 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7c9e2-a1c1-4a16-808c-8a8ffd618659 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:26.085289 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7c9e2-a1c1-4a16-808c-8a8ffd618659 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:26.085570 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7c9e2-a1c1-4a16-808c-8a8ffd618659 None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:26.090258 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7c9e2-a1c1-4a16-808c-8a8ffd618659 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:26.090258 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7c9e2-a1c1-4a16-808c-8a8ffd618659 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:26.143617 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 56/107] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Sep 1 16:20:25 2022] POST /identity/v3/projects => generated 302 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:20:26.148693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11bec22-d30e-49cb-9ef5-0033fc098e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:26.149511 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bec22-d30e-49cb-9ef5-0033fc098e18 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:26.149977 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bec22-d30e-49cb-9ef5-0033fc098e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:26.150415 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bec22-d30e-49cb-9ef5-0033fc098e18 None admin] PATH_INFO: `/v3/users/4955a8d9be064e8c8bddf4d21a8964f5` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:26.173045 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11bec22-d30e-49cb-9ef5-0033fc098e18 None admin] RBAC: Authorizing `identity:get_user(user_id=4955a8d9be064e8c8bddf4d21a8964f5)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:26.174914 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11bec22-d30e-49cb-9ef5-0033fc098e18 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:26.177358 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 52/108] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:25 2022] GET /identity/v3/users/4955a8d9be064e8c8bddf4d21a8964f5 => generated 289 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:26.183198 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7c9e2-a1c1-4a16-808c-8a8ffd618659 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:26.206300 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76ac61-1a87-4bbc-aa0d-09b2dbb8d53b None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:26.206965 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76ac61-1a87-4bbc-aa0d-09b2dbb8d53b None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:26.207469 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76ac61-1a87-4bbc-aa0d-09b2dbb8d53b None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:26.209034 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 53/109] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:26.214440 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa22a6-5961-463b-94dd-7b6949754deb None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:26.215025 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa22a6-5961-463b-94dd-7b6949754deb None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:26.215498 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa22a6-5961-463b-94dd-7b6949754deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:26.245070 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-e5fa22a6-5961-463b-94dd-7b6949754deb None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:26.246709 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-e5fa22a6-5961-463b-94dd-7b6949754deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:26.310373 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:26.326633 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:26.326823 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7c9e2-a1c1-4a16-808c-8a8ffd618659 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:26.391183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 54/110] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:20:26.392154 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa22a6-5961-463b-94dd-7b6949754deb None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:26.433043 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87df250-49af-49f8-bf56-c4b6f2946a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:26.434024 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87df250-49af-49f8-bf56-c4b6f2946a01 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:26.434539 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87df250-49af-49f8-bf56-c4b6f2946a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:26.435037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87df250-49af-49f8-bf56-c4b6f2946a01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:26.505567 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa22a6-5961-463b-94dd-7b6949754deb None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:26.549005 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b993d87-5105-4b4f-b432-0c5cf4cd769d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:26.549670 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b993d87-5105-4b4f-b432-0c5cf4cd769d None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:26.549958 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b993d87-5105-4b4f-b432-0c5cf4cd769d None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:26.550255 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b993d87-5105-4b4f-b432-0c5cf4cd769d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:26.572083 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b993d87-5105-4b4f-b432-0c5cf4cd769d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:26.578053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-2b993d87-5105-4b4f-b432-0c5cf4cd769d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:20:26.578053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:26.578053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:26.578053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:26.578053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:26.578053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:26.578053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:26.578053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:26.578053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:26.578053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:26.578053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:26.578053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:26.578053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:26.578053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:26.578053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:26.578053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:26.578908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:26.578908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:26.578908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:26.578908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:26.578908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:26.578908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:26.578908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:26.578908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:26.578908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:26.578908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:26.578908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:26.578908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:26.578908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:26.578908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:26.578908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:26.578908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:26.579628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:26.579628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:26.579628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:26.579628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:26.579628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:26.579628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:26.579628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:26.579628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:26.579628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:26.579628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:26.579628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:20:26.579628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:20:26.582426 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 55/111] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:26.583888 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87df250-49af-49f8-bf56-c4b6f2946a01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:26.584306 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ae5ab-368e-4d09-aef7-a89db3af384d None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:26.584602 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ae5ab-368e-4d09-aef7-a89db3af384d None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:26.584911 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ae5ab-368e-4d09-aef7-a89db3af384d None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:26.585439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87df250-49af-49f8-bf56-c4b6f2946a01 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:26.586144 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 56/112] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:26.592828 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222ae5ab-368e-4d09-aef7-a89db3af384d None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:26.621656 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:26.697665 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:26.711587 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222ae5ab-368e-4d09-aef7-a89db3af384d None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:26.714733 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 57/113] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:20:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:20:26.719913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f4cd6-bd9d-482b-a6f9-09f89c17c6e5 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:26.720210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f4cd6-bd9d-482b-a6f9-09f89c17c6e5 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:26.720486 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f4cd6-bd9d-482b-a6f9-09f89c17c6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:26.747586 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-791f4cd6-bd9d-482b-a6f9-09f89c17c6e5 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:26.750745 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-791f4cd6-bd9d-482b-a6f9-09f89c17c6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:26.759859 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f6d04-9066-4b43-b264-5c2904cb71a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:26.761560 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f6d04-9066-4b43-b264-5c2904cb71a6 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:26.762085 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f6d04-9066-4b43-b264-5c2904cb71a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:26.762642 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f6d04-9066-4b43-b264-5c2904cb71a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:26.764435 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f6d04-9066-4b43-b264-5c2904cb71a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:26.765553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f6d04-9066-4b43-b264-5c2904cb71a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:26.768206 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f6d04-9066-4b43-b264-5c2904cb71a6 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:26.781126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 57/114] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 1 16:20:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:20:26.782407 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f6d04-9066-4b43-b264-5c2904cb71a6 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:26.832741 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 58/115] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:26.833521 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f4cd6-bd9d-482b-a6f9-09f89c17c6e5 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:26.871475 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:26.889325 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f6d04-9066-4b43-b264-5c2904cb71a6 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:26.933629 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f72666-8184-4e38-8c8a-2bb9ddf0b912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:26.934645 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f72666-8184-4e38-8c8a-2bb9ddf0b912 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:26.935319 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f72666-8184-4e38-8c8a-2bb9ddf0b912 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:26.936935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f72666-8184-4e38-8c8a-2bb9ddf0b912 None admin] PATH_INFO: `/v3/projects/a3f472f7ed974397b7020b42b4f22bd3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:26.939079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f72666-8184-4e38-8c8a-2bb9ddf0b912 None admin] RBAC: Authorizing `identity:get_project(project_id=a3f472f7ed974397b7020b42b4f22bd3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:26.941369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f72666-8184-4e38-8c8a-2bb9ddf0b912 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:26.943409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 58/116] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:26 2022] GET /identity/v3/projects/a3f472f7ed974397b7020b42b4f22bd3 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:26.945443 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f72666-8184-4e38-8c8a-2bb9ddf0b912 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:27.058270 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f4cd6-bd9d-482b-a6f9-09f89c17c6e5 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:27.061396 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f72666-8184-4e38-8c8a-2bb9ddf0b912 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:27.075507 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:27.102693 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f25a17c-9ea5-401e-bf24-a25f34617a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:27.103322 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25a17c-9ea5-401e-bf24-a25f34617a4b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:27.103541 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25a17c-9ea5-401e-bf24-a25f34617a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:27.103824 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25a17c-9ea5-401e-bf24-a25f34617a4b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:27.111961 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2499dec9-07c4-4a8c-875a-96b6b3fb5247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:27.114973 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2499dec9-07c4-4a8c-875a-96b6b3fb5247 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:27.115563 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2499dec9-07c4-4a8c-875a-96b6b3fb5247 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:27.116094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2499dec9-07c4-4a8c-875a-96b6b3fb5247 None admin] PATH_INFO: `/v3/users/6d995078a79f4035bcbc04685a0579e2` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:27.118526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2499dec9-07c4-4a8c-875a-96b6b3fb5247 None admin] RBAC: Authorizing `identity:get_user(user_id=6d995078a79f4035bcbc04685a0579e2)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:27.120377 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2499dec9-07c4-4a8c-875a-96b6b3fb5247 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:27.123873 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 59/117] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:26 2022] GET /identity/v3/users/6d995078a79f4035bcbc04685a0579e2 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:27.125440 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2499dec9-07c4-4a8c-875a-96b6b3fb5247 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:27.126881 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f25a17c-9ea5-401e-bf24-a25f34617a4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:27.133265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-7f25a17c-9ea5-401e-bf24-a25f34617a4b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:20:27.133265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:27.133265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:27.133265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:27.133265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:27.133265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:27.133265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:27.133265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:27.133265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:27.133265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:27.133265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:27.133265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:27.133265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:27.133265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:27.133265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:27.133265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:27.134095 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:27.134095 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:27.134095 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:27.134095 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:27.134095 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:27.134095 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:27.134095 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:27.134095 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:27.134095 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:27.134095 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:27.134095 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:27.134095 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:27.134095 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:27.134095 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:27.134095 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:27.134095 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:27.134856 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:27.134856 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:27.134856 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:27.134856 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:27.134856 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:27.134856 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:27.134856 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:27.134856 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:27.134856 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:27.134856 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:27.134856 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:20:27.134856 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:20:27.137635 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 59/118] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 1 16:20:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:27.144354 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e3e94-7773-4419-8174-ade96351a833 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:27.144354 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e3e94-7773-4419-8174-ade96351a833 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:27.144354 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e3e94-7773-4419-8174-ade96351a833 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:27.145279 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 60/119] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:27.145830 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e3e94-7773-4419-8174-ade96351a833 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:27.159630 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:27.240068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2499dec9-07c4-4a8c-875a-96b6b3fb5247 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:27.254175 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e3e94-7773-4419-8174-ade96351a833 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:27.277100 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fae280b-96c7-4ca9-bd32-9e52f89d0b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:27.277721 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae280b-96c7-4ca9-bd32-9e52f89d0b5a None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:27.277999 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae280b-96c7-4ca9-bd32-9e52f89d0b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:27.278299 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae280b-96c7-4ca9-bd32-9e52f89d0b5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:27.279029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fae280b-96c7-4ca9-bd32-9e52f89d0b5a None admin] RBAC: Adding query filter params (role.id=12fe0aa7f1174edfb109315b88d6267b, scope.project.id=a3f472f7ed974397b7020b42b4f22bd3, user.id=4955a8d9be064e8c8bddf4d21a8964f5) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:27.279340 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fae280b-96c7-4ca9-bd32-9e52f89d0b5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:27.280833 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fae280b-96c7-4ca9-bd32-9e52f89d0b5a None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:27.298289 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 60/120] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:20:27 2022] GET /identity/v3/role_assignments?user.id=4955a8d9be064e8c8bddf4d21a8964f5&scope.project.id=a3f472f7ed974397b7020b42b4f22bd3&role.id=12fe0aa7f1174edfb109315b88d6267b => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:27.298289 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fae280b-96c7-4ca9-bd32-9e52f89d0b5a None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:27.300732 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74935839-1fe4-4fea-8e32-a14f3f41e01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:27.301528 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74935839-1fe4-4fea-8e32-a14f3f41e01a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:27.301824 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74935839-1fe4-4fea-8e32-a14f3f41e01a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:27.302178 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74935839-1fe4-4fea-8e32-a14f3f41e01a None admin] PATH_INFO: `/v3/projects/a3f472f7ed974397b7020b42b4f22bd3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:27.303519 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74935839-1fe4-4fea-8e32-a14f3f41e01a None admin] RBAC: Authorizing `identity:get_project(project_id=a3f472f7ed974397b7020b42b4f22bd3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:27.305167 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74935839-1fe4-4fea-8e32-a14f3f41e01a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:27.312179 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 61/121] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:27 2022] GET /identity/v3/projects/a3f472f7ed974397b7020b42b4f22bd3 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:27.312179 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74935839-1fe4-4fea-8e32-a14f3f41e01a None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:27.372457 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:27.403701 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:27.424098 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fae280b-96c7-4ca9-bd32-9e52f89d0b5a None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:27.448614 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74935839-1fe4-4fea-8e32-a14f3f41e01a None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:27.464106 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574bfc0e-98f7-48f8-9108-fc8e6874c57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:27.464818 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bfc0e-98f7-48f8-9108-fc8e6874c57c None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:27.465106 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bfc0e-98f7-48f8-9108-fc8e6874c57c None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:27.465386 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bfc0e-98f7-48f8-9108-fc8e6874c57c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:27.476633 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574bfc0e-98f7-48f8-9108-fc8e6874c57c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:27.482145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-574bfc0e-98f7-48f8-9108-fc8e6874c57c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:20:27.482145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:27.482145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:27.482145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:27.482145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:27.482145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:27.482145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:27.482145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:27.482145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:27.482145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:27.482145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:27.482145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:27.482145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:27.482145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:27.482145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:27.482145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:27.482988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:27.482988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:27.482988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:27.482988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:27.482988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:27.482988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:27.482988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:27.482988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:27.482988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:27.482988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:27.482988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:27.482988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:27.482988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:27.482988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:27.482988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:27.482988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:27.484242 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:27.484242 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:27.484242 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:27.484242 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:27.484242 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:27.484242 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:27.484242 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:27.484242 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:27.484242 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:27.484242 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:27.484242 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:20:27.484242 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:20:27.484242 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 61/122] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:20:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:27.490093 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448168c-3ca7-472a-8543-922b1428807e None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:27.490093 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448168c-3ca7-472a-8543-922b1428807e None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:27.490093 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448168c-3ca7-472a-8543-922b1428807e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:27.504926 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-0448168c-3ca7-472a-8543-922b1428807e None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:27.506494 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-0448168c-3ca7-472a-8543-922b1428807e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:27.507541 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcfe244-7409-4e3a-b2e9-5489c9588b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:27.508380 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfe244-7409-4e3a-b2e9-5489c9588b87 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:27.509034 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfe244-7409-4e3a-b2e9-5489c9588b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:27.509501 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfe244-7409-4e3a-b2e9-5489c9588b87 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:27.512481 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcfe244-7409-4e3a-b2e9-5489c9588b87 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:27.512841 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcfe244-7409-4e3a-b2e9-5489c9588b87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:27.515297 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcfe244-7409-4e3a-b2e9-5489c9588b87 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:27.532043 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 62/123] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:20:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:27.533114 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcfe244-7409-4e3a-b2e9-5489c9588b87 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:27.587260 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 62/124] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:27.588145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448168c-3ca7-472a-8543-922b1428807e None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:27.656207 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcfe244-7409-4e3a-b2e9-5489c9588b87 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:27.665826 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:27.685298 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:27.695522 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448168c-3ca7-472a-8543-922b1428807e None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:27.719430 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f6169-ea05-494a-8d69-a2a90c07cd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:27.720109 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f6169-ea05-494a-8d69-a2a90c07cd8c None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:27.720395 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f6169-ea05-494a-8d69-a2a90c07cd8c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:27.720902 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f6169-ea05-494a-8d69-a2a90c07cd8c None admin] PATH_INFO: `/v3/projects/a3f472f7ed974397b7020b42b4f22bd3/users/6d995078a79f4035bcbc04685a0579e2/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:27.726988 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4f6169-ea05-494a-8d69-a2a90c07cd8c None admin] RBAC: Authorizing `identity:create_grant(project_id=a3f472f7ed974397b7020b42b4f22bd3, user_id=6d995078a79f4035bcbc04685a0579e2, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:27.728495 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4f6169-ea05-494a-8d69-a2a90c07cd8c None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:27.731908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06351818-51c5-4362-99fc-066e6855f7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:27.733180 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06351818-51c5-4362-99fc-066e6855f7fc None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:27.734269 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06351818-51c5-4362-99fc-066e6855f7fc None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:27.734778 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06351818-51c5-4362-99fc-066e6855f7fc None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:27.735705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06351818-51c5-4362-99fc-066e6855f7fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:27.736200 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06351818-51c5-4362-99fc-066e6855f7fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:27.737811 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06351818-51c5-4362-99fc-066e6855f7fc None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:27.747205 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 63/125] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:27.753571 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06351818-51c5-4362-99fc-066e6855f7fc None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:27.765302 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 63/126] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 1 16:20:27 2022] PUT /identity/v3/projects/a3f472f7ed974397b7020b42b4f22bd3/users/6d995078a79f4035bcbc04685a0579e2/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:20:27.766226 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f6169-ea05-494a-8d69-a2a90c07cd8c None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:27.870725 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06351818-51c5-4362-99fc-066e6855f7fc None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:27.910079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea5afaf-8b0d-4517-a782-d105b78cfcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:27.911114 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5afaf-8b0d-4517-a782-d105b78cfcc0 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:27.911591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5afaf-8b0d-4517-a782-d105b78cfcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:27.912138 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5afaf-8b0d-4517-a782-d105b78cfcc0 None admin] PATH_INFO: `/v3/users/6d995078a79f4035bcbc04685a0579e2` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:27.913495 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea5afaf-8b0d-4517-a782-d105b78cfcc0 None admin] RBAC: Authorizing `identity:get_user(user_id=6d995078a79f4035bcbc04685a0579e2)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:27.915425 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea5afaf-8b0d-4517-a782-d105b78cfcc0 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:27.917082 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 64/127] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:27 2022] GET /identity/v3/users/6d995078a79f4035bcbc04685a0579e2 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:27.922637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea5afaf-8b0d-4517-a782-d105b78cfcc0 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:27.930267 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:28.019454 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f6169-ea05-494a-8d69-a2a90c07cd8c None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:28.059124 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ddf12-0300-4713-a36a-ce0b66cb39d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:28.060172 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ddf12-0300-4713-a36a-ce0b66cb39d8 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:28.060458 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ddf12-0300-4713-a36a-ce0b66cb39d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:28.060737 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ddf12-0300-4713-a36a-ce0b66cb39d8 None admin] PATH_INFO: `/v3/users/eed887c1c93d4f24959b0f90e215c424` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:28.062368 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61ddf12-0300-4713-a36a-ce0b66cb39d8 None admin] RBAC: Authorizing `identity:get_user(user_id=eed887c1c93d4f24959b0f90e215c424)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:28.063057 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea5afaf-8b0d-4517-a782-d105b78cfcc0 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:28.063955 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61ddf12-0300-4713-a36a-ce0b66cb39d8 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:28.065870 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 64/128] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:27 2022] GET /identity/v3/users/eed887c1c93d4f24959b0f90e215c424 => generated 247 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:28.067764 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ddf12-0300-4713-a36a-ce0b66cb39d8 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:28.102306 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b50c7c-a32c-40d1-93d3-941d9beecea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:28.102957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b50c7c-a32c-40d1-93d3-941d9beecea9 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:28.103244 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b50c7c-a32c-40d1-93d3-941d9beecea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:28.103579 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b50c7c-a32c-40d1-93d3-941d9beecea9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:28.112602 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:28.135157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b50c7c-a32c-40d1-93d3-941d9beecea9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:28.141027 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-19b50c7c-a32c-40d1-93d3-941d9beecea9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:20:28.141027 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:28.141027 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:28.141027 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:28.141027 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:28.141027 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:28.141027 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:28.141027 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:28.141027 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:28.141027 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:28.141027 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:28.141027 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:28.141027 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:28.141027 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:28.141027 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:28.141027 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:28.141863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:28.141863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:28.141863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:28.141863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:28.141863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:28.141863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:28.141863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:28.141863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:28.141863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:28.141863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:28.141863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:28.141863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:28.141863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:28.141863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:28.141863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:28.141863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:28.142725 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:28.142725 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:28.142725 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:28.142725 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:28.142725 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:28.142725 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:28.142725 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:28.142725 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:28.142725 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:28.142725 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:28.142725 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:20:28.142725 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:20:28.142725 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 65/129] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:28.151191 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b50c7c-a32c-40d1-93d3-941d9beecea9 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:28.191709 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ddf12-0300-4713-a36a-ce0b66cb39d8 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:28.242914 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:28.243616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa250a-0504-4632-901f-7f11712c4331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:28.244273 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa250a-0504-4632-901f-7f11712c4331 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:28.244580 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa250a-0504-4632-901f-7f11712c4331 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:28.245591 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa250a-0504-4632-901f-7f11712c4331 None admin] PATH_INFO: `/v3/projects/a3f472f7ed974397b7020b42b4f22bd3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:28.246036 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa250a-0504-4632-901f-7f11712c4331 None admin] RBAC: Authorizing `identity:get_project(project_id=a3f472f7ed974397b7020b42b4f22bd3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:28.247520 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa250a-0504-4632-901f-7f11712c4331 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:28.249515 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 65/130] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:28 2022] GET /identity/v3/projects/a3f472f7ed974397b7020b42b4f22bd3 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:28.250342 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa250a-0504-4632-901f-7f11712c4331 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:28.288111 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b50c7c-a32c-40d1-93d3-941d9beecea9 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:28.336656 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60cfae-7d3a-4492-9381-845f846fca8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:28.337362 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60cfae-7d3a-4492-9381-845f846fca8e None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:28.337749 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60cfae-7d3a-4492-9381-845f846fca8e None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:28.338071 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60cfae-7d3a-4492-9381-845f846fca8e None admin] PATH_INFO: `/v3/projects/a3f472f7ed974397b7020b42b4f22bd3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:28.339323 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60cfae-7d3a-4492-9381-845f846fca8e None admin] RBAC: Authorizing `identity:get_project(project_id=a3f472f7ed974397b7020b42b4f22bd3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:28.347175 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60cfae-7d3a-4492-9381-845f846fca8e None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:28.348988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 66/131] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:28 2022] GET /identity/v3/projects/a3f472f7ed974397b7020b42b4f22bd3 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:28.349993 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60cfae-7d3a-4492-9381-845f846fca8e None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:28.356430 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa250a-0504-4632-901f-7f11712c4331 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:28.358988 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:28.392111 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ddf1a-bfbe-42c0-a47d-bf3690884582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:28.392775 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ddf1a-bfbe-42c0-a47d-bf3690884582 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:28.393077 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ddf1a-bfbe-42c0-a47d-bf3690884582 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:28.393411 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ddf1a-bfbe-42c0-a47d-bf3690884582 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:28.394094 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ddf1a-bfbe-42c0-a47d-bf3690884582 None admin] RBAC: Adding query filter params (name=member) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:28.394515 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ddf1a-bfbe-42c0-a47d-bf3690884582 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:28.395984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ddf1a-bfbe-42c0-a47d-bf3690884582 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:28.405265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 66/132] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 1 16:20:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:28.406553 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ddf1a-bfbe-42c0-a47d-bf3690884582 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:28.551414 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60cfae-7d3a-4492-9381-845f846fca8e None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:28.567779 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ddf1a-bfbe-42c0-a47d-bf3690884582 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:28.605339 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:28.613479 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab8afd-893d-4083-bdc0-dd26e0d64703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:28.614153 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab8afd-893d-4083-bdc0-dd26e0d64703 None admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:28.614474 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab8afd-893d-4083-bdc0-dd26e0d64703 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:28.614751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab8afd-893d-4083-bdc0-dd26e0d64703 None admin] PATH_INFO: `/v3/projects/a3f472f7ed974397b7020b42b4f22bd3/users/6d995078a79f4035bcbc04685a0579e2/roles/f2a6e936aa294323ba7087bf377276b2` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:28.617051 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ab8afd-893d-4083-bdc0-dd26e0d64703 None admin] RBAC: Authorizing `identity:create_grant(project_id=a3f472f7ed974397b7020b42b4f22bd3, user_id=6d995078a79f4035bcbc04685a0579e2, role_id=f2a6e936aa294323ba7087bf377276b2)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:28.618720 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ab8afd-893d-4083-bdc0-dd26e0d64703 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:28.636071 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 67/133] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 1 16:20:28 2022] PUT /identity/v3/projects/a3f472f7ed974397b7020b42b4f22bd3/users/6d995078a79f4035bcbc04685a0579e2/roles/f2a6e936aa294323ba7087bf377276b2 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:20:28.636851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab8afd-893d-4083-bdc0-dd26e0d64703 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:28.640434 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef579306-f26d-45f6-a5b8-e1f78cb87ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:28.641133 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef579306-f26d-45f6-a5b8-e1f78cb87ddc None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:28.641422 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef579306-f26d-45f6-a5b8-e1f78cb87ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:28.641699 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef579306-f26d-45f6-a5b8-e1f78cb87ddc None admin] PATH_INFO: `/v3/projects/a3f472f7ed974397b7020b42b4f22bd3/users/eed887c1c93d4f24959b0f90e215c424/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:28.647837 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef579306-f26d-45f6-a5b8-e1f78cb87ddc None admin] RBAC: Authorizing `identity:create_grant(project_id=a3f472f7ed974397b7020b42b4f22bd3, user_id=eed887c1c93d4f24959b0f90e215c424, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:28.649327 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef579306-f26d-45f6-a5b8-e1f78cb87ddc None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:28.689733 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 67/134] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 1 16:20:28 2022] PUT /identity/v3/projects/a3f472f7ed974397b7020b42b4f22bd3/users/eed887c1c93d4f24959b0f90e215c424/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:20:28.789697 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab8afd-893d-4083-bdc0-dd26e0d64703 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:28.845044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f042a48d-cfcc-495e-bccf-4f715d6e7d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:28.845633 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f042a48d-cfcc-495e-bccf-4f715d6e7d5e None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:28.845924 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f042a48d-cfcc-495e-bccf-4f715d6e7d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:28.846204 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f042a48d-cfcc-495e-bccf-4f715d6e7d5e None admin] PATH_INFO: `/v3/users/6d995078a79f4035bcbc04685a0579e2` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:28.847630 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f042a48d-cfcc-495e-bccf-4f715d6e7d5e None admin] RBAC: Authorizing `identity:get_user(user_id=6d995078a79f4035bcbc04685a0579e2)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:28.849112 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f042a48d-cfcc-495e-bccf-4f715d6e7d5e None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:28.851033 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 68/135] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:28 2022] GET /identity/v3/users/6d995078a79f4035bcbc04685a0579e2 => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:28.856720 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef579306-f26d-45f6-a5b8-e1f78cb87ddc None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:28.896092 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:28.975988 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef579306-f26d-45f6-a5b8-e1f78cb87ddc None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:29.020372 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a59734-95c7-434b-84aa-387ea846d356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:29.021026 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a59734-95c7-434b-84aa-387ea846d356 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:29.021309 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a59734-95c7-434b-84aa-387ea846d356 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:29.021589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a59734-95c7-434b-84aa-387ea846d356 None admin] PATH_INFO: `/v3/projects/1cf23dd85509483f9d433c73e9329b89` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:29.022773 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a59734-95c7-434b-84aa-387ea846d356 None admin] RBAC: Authorizing `identity:get_project(project_id=1cf23dd85509483f9d433c73e9329b89)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:29.024233 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a59734-95c7-434b-84aa-387ea846d356 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:29.026178 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 68/136] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:28 2022] GET /identity/v3/projects/1cf23dd85509483f9d433c73e9329b89 => generated 312 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:29.029076 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c470e1-317a-4710-9348-4e01c335f6ac None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:29.029373 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c470e1-317a-4710-9348-4e01c335f6ac None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:29.029655 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c470e1-317a-4710-9348-4e01c335f6ac None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:29.031189 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 69/137] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:29.031778 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a59734-95c7-434b-84aa-387ea846d356 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:29.037765 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667d1b3-4f6f-4efb-b4af-197dc77b59a6 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:29.038053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667d1b3-4f6f-4efb-b4af-197dc77b59a6 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:29.038363 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667d1b3-4f6f-4efb-b4af-197dc77b59a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:29.058721 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-0667d1b3-4f6f-4efb-b4af-197dc77b59a6 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:29.060042 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-0667d1b3-4f6f-4efb-b4af-197dc77b59a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:29.126188 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:29.134602 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 70/138] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:20:29.209457 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:29.216903 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a59734-95c7-434b-84aa-387ea846d356 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:29.264164 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc6e77-2281-4650-b8f3-60f924f0fe18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:29.264876 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc6e77-2281-4650-b8f3-60f924f0fe18 None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:29.265163 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc6e77-2281-4650-b8f3-60f924f0fe18 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:29.265443 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc6e77-2281-4650-b8f3-60f924f0fe18 None admin] PATH_INFO: `/v3/projects/1cf23dd85509483f9d433c73e9329b89/users/6d995078a79f4035bcbc04685a0579e2/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:29.267407 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbc6e77-2281-4650-b8f3-60f924f0fe18 None admin] RBAC: Authorizing `identity:create_grant(project_id=1cf23dd85509483f9d433c73e9329b89, user_id=6d995078a79f4035bcbc04685a0579e2, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:29.268874 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbc6e77-2281-4650-b8f3-60f924f0fe18 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:29.291093 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 69/139] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 1 16:20:29 2022] PUT /identity/v3/projects/1cf23dd85509483f9d433c73e9329b89/users/6d995078a79f4035bcbc04685a0579e2/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:20:29.525417 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667d1b3-4f6f-4efb-b4af-197dc77b59a6 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:29.641215 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667d1b3-4f6f-4efb-b4af-197dc77b59a6 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:29.680558 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40543ade-8fcc-424f-96af-1c75b483da9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:29.681209 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40543ade-8fcc-424f-96af-1c75b483da9c None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:29.681496 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40543ade-8fcc-424f-96af-1c75b483da9c None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:29.681773 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40543ade-8fcc-424f-96af-1c75b483da9c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:29.682899 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40543ade-8fcc-424f-96af-1c75b483da9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:29.684364 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40543ade-8fcc-424f-96af-1c75b483da9c None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:29.687835 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 71/140] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:20:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:29.695324 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc6e77-2281-4650-b8f3-60f924f0fe18 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:29.806538 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94cafa-28a4-4e9c-a0b7-97a8c2cc7b08 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:29.807180 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94cafa-28a4-4e9c-a0b7-97a8c2cc7b08 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:29.807180 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94cafa-28a4-4e9c-a0b7-97a8c2cc7b08 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:29.809747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 72/141] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:29.814634 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e33da7-42b2-4d93-9c96-5b6af8ee5db0 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:29.815151 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e33da7-42b2-4d93-9c96-5b6af8ee5db0 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:29.815618 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e33da7-42b2-4d93-9c96-5b6af8ee5db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:29.835485 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-b3e33da7-42b2-4d93-9c96-5b6af8ee5db0 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:29.837056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-b3e33da7-42b2-4d93-9c96-5b6af8ee5db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:29.845834 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc6e77-2281-4650-b8f3-60f924f0fe18 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:29.932435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c297b5d8-aa52-4126-920b-8d635116402d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:29.933055 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297b5d8-aa52-4126-920b-8d635116402d None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:29.933346 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297b5d8-aa52-4126-920b-8d635116402d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:29.933629 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297b5d8-aa52-4126-920b-8d635116402d None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:29.934314 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c297b5d8-aa52-4126-920b-8d635116402d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:29.935780 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c297b5d8-aa52-4126-920b-8d635116402d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:29.957479 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:29.984614 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 73/142] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:20:29.989688 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 70/143] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 1 16:20:29 2022] POST /identity/v3/users => generated 283 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:20:30.384853 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e33da7-42b2-4d93-9c96-5b6af8ee5db0 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:30.517966 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e33da7-42b2-4d93-9c96-5b6af8ee5db0 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:30.560457 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773ec15-9a70-43a7-97de-88bc9b8d3d38 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:30.560787 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773ec15-9a70-43a7-97de-88bc9b8d3d38 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:30.561105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773ec15-9a70-43a7-97de-88bc9b8d3d38 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:30.563059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 71/144] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:20:30.568091 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde6e89-cf59-4d93-9c5c-29192371b71b None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:30.568419 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde6e89-cf59-4d93-9c5c-29192371b71b None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:30.568736 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde6e89-cf59-4d93-9c5c-29192371b71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:30.575020 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a43fd7-0192-4f74-99c3-f5daefcb7620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:30.575664 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a43fd7-0192-4f74-99c3-f5daefcb7620 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:30.575946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a43fd7-0192-4f74-99c3-f5daefcb7620 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:30.576211 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a43fd7-0192-4f74-99c3-f5daefcb7620 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:30.597685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-8fde6e89-cf59-4d93-9c5c-29192371b71b None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:30.603120 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8fde6e89-cf59-4d93-9c5c-29192371b71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:30.605938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a43fd7-0192-4f74-99c3-f5daefcb7620 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:30.611367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-02a43fd7-0192-4f74-99c3-f5daefcb7620 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:20:30.611367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:30.611367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:30.611367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:30.611367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:30.611367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:30.611367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:30.611367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:30.611367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:30.611367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:30.611367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:30.611367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:30.611367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:30.611367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:30.611367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:30.611367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:30.612248 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:30.612248 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:30.612248 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:30.612248 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:30.612248 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:30.612248 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:30.612248 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:30.612248 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:30.612248 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:30.612248 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:30.612248 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:30.612248 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:30.612248 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:30.612248 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:30.612248 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:30.612248 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:30.612971 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:30.612971 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:30.612971 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:30.612971 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:30.612971 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:30.612971 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:30.612971 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:30.612971 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:30.612971 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:30.612971 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:30.612971 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:20:30.612971 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:20:30.612971 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 74/145] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:30.614626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:30.622713 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a43fd7-0192-4f74-99c3-f5daefcb7620 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:30.687292 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:30.744145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a43fd7-0192-4f74-99c3-f5daefcb7620 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:30.758007 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 72/146] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:20:30.782180 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d60366-c2be-41d6-aae2-790f3f6080fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:30.783153 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d60366-c2be-41d6-aae2-790f3f6080fe None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:30.783682 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d60366-c2be-41d6-aae2-790f3f6080fe None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:30.784145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d60366-c2be-41d6-aae2-790f3f6080fe None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:30.785053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d60366-c2be-41d6-aae2-790f3f6080fe None admin] RBAC: Adding query filter params (name=reader) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:30.785583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d60366-c2be-41d6-aae2-790f3f6080fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:30.787247 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d60366-c2be-41d6-aae2-790f3f6080fe None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:30.804513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 75/147] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 1 16:20:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:30.812664 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde6e89-cf59-4d93-9c5c-29192371b71b None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:30.962668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde6e89-cf59-4d93-9c5c-29192371b71b None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:31.011620 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa725b0-390f-48fb-9116-a153265543dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:31.012279 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa725b0-390f-48fb-9116-a153265543dc None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:31.012566 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa725b0-390f-48fb-9116-a153265543dc None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:31.012846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa725b0-390f-48fb-9116-a153265543dc None admin] PATH_INFO: `/v3/users/4955a8d9be064e8c8bddf4d21a8964f5` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:31.018952 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa725b0-390f-48fb-9116-a153265543dc None admin] RBAC: Authorizing `identity:get_user(user_id=4955a8d9be064e8c8bddf4d21a8964f5)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:31.020431 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa725b0-390f-48fb-9116-a153265543dc None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:31.022295 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 73/148] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:30 2022] GET /identity/v3/users/4955a8d9be064e8c8bddf4d21a8964f5 => generated 289 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:31.028153 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d60366-c2be-41d6-aae2-790f3f6080fe None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:31.156509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d60366-c2be-41d6-aae2-790f3f6080fe None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:31.169400 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:31.200140 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff256e8-3696-4044-a9c4-ecafb497a174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:31.201161 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff256e8-3696-4044-a9c4-ecafb497a174 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:31.201697 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff256e8-3696-4044-a9c4-ecafb497a174 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:31.202764 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff256e8-3696-4044-a9c4-ecafb497a174 None admin] PATH_INFO: `/v3/projects/a3f472f7ed974397b7020b42b4f22bd3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:31.204082 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff256e8-3696-4044-a9c4-ecafb497a174 None admin] RBAC: Authorizing `identity:get_project(project_id=a3f472f7ed974397b7020b42b4f22bd3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:31.205727 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff256e8-3696-4044-a9c4-ecafb497a174 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:31.207570 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 76/149] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:31 2022] GET /identity/v3/projects/a3f472f7ed974397b7020b42b4f22bd3 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:31.219022 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa725b0-390f-48fb-9116-a153265543dc None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:31.219661 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff256e8-3696-4044-a9c4-ecafb497a174 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:31.240767 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:31.368895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff256e8-3696-4044-a9c4-ecafb497a174 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:31.439355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa725b0-390f-48fb-9116-a153265543dc None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:31.446978 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b165a-bb1b-495a-8d09-aecc5c6866b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:31.447645 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b165a-bb1b-495a-8d09-aecc5c6866b4 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:31.447931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b165a-bb1b-495a-8d09-aecc5c6866b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:31.448210 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b165a-bb1b-495a-8d09-aecc5c6866b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:31.470066 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9b165a-bb1b-495a-8d09-aecc5c6866b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:31.475535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-ca9b165a-bb1b-495a-8d09-aecc5c6866b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:20:31.475535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:31.475535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:31.475535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:31.475535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:31.475535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:31.475535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:31.475535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:31.475535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:31.475535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:31.475535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:31.475535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:31.475535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:31.475535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:31.475535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:31.475535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:31.476351 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:31.476351 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:31.476351 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:31.476351 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:31.476351 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:31.476351 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:31.476351 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:31.476351 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:31.476351 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:31.476351 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:31.476351 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:31.476351 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:31.476351 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:31.476351 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:31.476351 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:31.476351 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:31.477094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:31.477094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:31.477094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:31.477094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:31.477094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:31.477094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:31.477094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:31.477094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:31.477094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:31.477094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:31.477094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:20:31.477094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:20:31.477094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 77/150] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:20:31.486055 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365b4ca-9d12-4ebb-9eaa-bfd4e748ee61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:31.486780 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365b4ca-9d12-4ebb-9eaa-bfd4e748ee61 None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:31.487064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365b4ca-9d12-4ebb-9eaa-bfd4e748ee61 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:31.487343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365b4ca-9d12-4ebb-9eaa-bfd4e748ee61 None admin] PATH_INFO: `/v3/projects/a3f472f7ed974397b7020b42b4f22bd3/users/4955a8d9be064e8c8bddf4d21a8964f5/roles/12fe0aa7f1174edfb109315b88d6267b` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:31.489381 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8365b4ca-9d12-4ebb-9eaa-bfd4e748ee61 None admin] RBAC: Authorizing `identity:create_grant(project_id=a3f472f7ed974397b7020b42b4f22bd3, user_id=4955a8d9be064e8c8bddf4d21a8964f5, role_id=12fe0aa7f1174edfb109315b88d6267b)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:31.490866 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8365b4ca-9d12-4ebb-9eaa-bfd4e748ee61 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:31.491271 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b165a-bb1b-495a-8d09-aecc5c6866b4 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:31.510957 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 74/151] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 1 16:20:31 2022] PUT /identity/v3/projects/a3f472f7ed974397b7020b42b4f22bd3/users/4955a8d9be064e8c8bddf4d21a8964f5/roles/12fe0aa7f1174edfb109315b88d6267b => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:20:31.512294 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1f089-a462-4cc8-ab0b-7c15623d3f4a None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:31.512621 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1f089-a462-4cc8-ab0b-7c15623d3f4a None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:31.512967 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1f089-a462-4cc8-ab0b-7c15623d3f4a None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:31.516032 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 75/152] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:31.521602 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fa656-efb9-4f11-a605-ef4c90796792 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:31.521992 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fa656-efb9-4f11-a605-ef4c90796792 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:31.522306 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fa656-efb9-4f11-a605-ef4c90796792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:31.543766 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-da1fa656-efb9-4f11-a605-ef4c90796792 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:31.545107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-da1fa656-efb9-4f11-a605-ef4c90796792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:31.549562 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:31.630292 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:31.634320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 76/153] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:20:31.649620 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b165a-bb1b-495a-8d09-aecc5c6866b4 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:31.687910 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b7316-70a8-46bb-ab40-5ecf441d290e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:31.688612 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b7316-70a8-46bb-ab40-5ecf441d290e None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:31.688871 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b7316-70a8-46bb-ab40-5ecf441d290e None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:31.689146 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b7316-70a8-46bb-ab40-5ecf441d290e None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:31.689824 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b7316-70a8-46bb-ab40-5ecf441d290e None admin] RBAC: Adding query filter params (name=member) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:31.690128 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b7316-70a8-46bb-ab40-5ecf441d290e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:31.691562 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b7316-70a8-46bb-ab40-5ecf441d290e None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:31.701368 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 78/154] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 1 16:20:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:31.707466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fa656-efb9-4f11-a605-ef4c90796792 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:31.836632 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fa656-efb9-4f11-a605-ef4c90796792 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:31.884764 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e49a7c-d6e8-458b-801a-1de7d724dc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:31.885421 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e49a7c-d6e8-458b-801a-1de7d724dc49 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:31.885699 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e49a7c-d6e8-458b-801a-1de7d724dc49 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:31.885967 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e49a7c-d6e8-458b-801a-1de7d724dc49 None admin] PATH_INFO: `/v3/users/6d995078a79f4035bcbc04685a0579e2` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:31.887134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e49a7c-d6e8-458b-801a-1de7d724dc49 None admin] RBAC: Authorizing `identity:get_user(user_id=6d995078a79f4035bcbc04685a0579e2)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:31.888577 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e49a7c-d6e8-458b-801a-1de7d724dc49 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:31.890969 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 77/155] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:31 2022] GET /identity/v3/users/6d995078a79f4035bcbc04685a0579e2 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:31.896381 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b7316-70a8-46bb-ab40-5ecf441d290e None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:32.016963 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:32.020434 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b7316-70a8-46bb-ab40-5ecf441d290e None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:32.063912 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d4c95-7314-49f0-a77f-1f5f6d4b01df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:32.064569 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d4c95-7314-49f0-a77f-1f5f6d4b01df None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:32.064921 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d4c95-7314-49f0-a77f-1f5f6d4b01df None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:32.065205 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d4c95-7314-49f0-a77f-1f5f6d4b01df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:32.144307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0d4c95-7314-49f0-a77f-1f5f6d4b01df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:32.145816 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0d4c95-7314-49f0-a77f-1f5f6d4b01df None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:32.231686 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e49a7c-d6e8-458b-801a-1de7d724dc49 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:32.263429 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:32.268947 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 79/156] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:20:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:20:32.274489 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d4c95-7314-49f0-a77f-1f5f6d4b01df None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:32.358858 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:32.409417 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e49a7c-d6e8-458b-801a-1de7d724dc49 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:32.449624 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d4c95-7314-49f0-a77f-1f5f6d4b01df None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:32.491002 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf48b2-5af7-4460-8afb-ab5100e217e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:32.491668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf48b2-5af7-4460-8afb-ab5100e217e9 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:32.491949 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf48b2-5af7-4460-8afb-ab5100e217e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:32.492218 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf48b2-5af7-4460-8afb-ab5100e217e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:32.505225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ce7bf-2100-4a7b-b68c-7a26c5b11690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:32.505877 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ce7bf-2100-4a7b-b68c-7a26c5b11690 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:32.506166 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ce7bf-2100-4a7b-b68c-7a26c5b11690 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:32.506455 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ce7bf-2100-4a7b-b68c-7a26c5b11690 None admin] PATH_INFO: `/v3/projects/a3f472f7ed974397b7020b42b4f22bd3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:32.507621 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ce7bf-2100-4a7b-b68c-7a26c5b11690 None admin] RBAC: Authorizing `identity:get_project(project_id=a3f472f7ed974397b7020b42b4f22bd3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:32.509180 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ce7bf-2100-4a7b-b68c-7a26c5b11690 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:32.511081 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 80/157] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:32 2022] GET /identity/v3/projects/a3f472f7ed974397b7020b42b4f22bd3 => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:20:32.511997 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027edacb-b415-4b14-96f5-0058dc52db72 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:32.512281 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027edacb-b415-4b14-96f5-0058dc52db72 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:32.512558 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027edacb-b415-4b14-96f5-0058dc52db72 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:32.518603 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 81/158] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:32.520288 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027edacb-b415-4b14-96f5-0058dc52db72 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:32.523707 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf48b2-5af7-4460-8afb-ab5100e217e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:32.528573 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-e1cf48b2-5af7-4460-8afb-ab5100e217e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:20:32.528573 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:32.528573 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:32.528573 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:32.528573 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:32.528573 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:32.528573 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:32.528573 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:32.528573 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:32.528573 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:32.528573 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:32.528573 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:32.528573 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:32.528573 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:32.528573 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:32.528573 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:32.529468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:32.529468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:32.529468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:32.529468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:32.529468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:32.529468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:32.529468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:32.529468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:32.529468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:32.529468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:32.529468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:32.529468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:32.529468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:32.529468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:32.529468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:32.529468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:32.530259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:32.530259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:32.530259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:32.530259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:32.530259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:32.530259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:32.530259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:32.530259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:32.530259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:32.530259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:32.530259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:20:32.530259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:20:32.530259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 78/159] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 1 16:20:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:20:32.536250 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa449e-8bf9-40a9-8218-5270f4434cd0 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:32.536698 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa449e-8bf9-40a9-8218-5270f4434cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:32.536982 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa449e-8bf9-40a9-8218-5270f4434cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:32.576386 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-ecfa449e-8bf9-40a9-8218-5270f4434cd0 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:32.578205 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ecfa449e-8bf9-40a9-8218-5270f4434cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:32.674498 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:32.760800 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 79/160] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:32.761679 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa449e-8bf9-40a9-8218-5270f4434cd0 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:32.770910 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027edacb-b415-4b14-96f5-0058dc52db72 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:32.844076 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ca1c0-9ce0-41ad-86be-28ed68f05294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:32.844776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ca1c0-9ce0-41ad-86be-28ed68f05294 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:32.845314 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ca1c0-9ce0-41ad-86be-28ed68f05294 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:32.845756 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ca1c0-9ce0-41ad-86be-28ed68f05294 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:32.846791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ca1c0-9ce0-41ad-86be-28ed68f05294 None admin] RBAC: Adding query filter params (role.id=45545334e1fc473c9d85fc91a9b0a35c, scope.project.id=a3f472f7ed974397b7020b42b4f22bd3, user.id=6d995078a79f4035bcbc04685a0579e2) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:32.847102 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ca1c0-9ce0-41ad-86be-28ed68f05294 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:32.849154 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ca1c0-9ce0-41ad-86be-28ed68f05294 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:32.865583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 82/161] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:20:32 2022] GET /identity/v3/role_assignments?user.id=6d995078a79f4035bcbc04685a0579e2&scope.project.id=a3f472f7ed974397b7020b42b4f22bd3&role.id=45545334e1fc473c9d85fc91a9b0a35c => generated 608 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:20:32.866931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f2c66-d70e-4cc9-9992-552f1dcc28c7 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:32.867218 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f2c66-d70e-4cc9-9992-552f1dcc28c7 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:32.867496 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f2c66-d70e-4cc9-9992-552f1dcc28c7 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:32.868836 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 83/162] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:32.875576 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069ecab-11e5-41be-bc67-dbb84830436b None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:32.875857 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069ecab-11e5-41be-bc67-dbb84830436b None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:32.876133 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069ecab-11e5-41be-bc67-dbb84830436b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:32.877215 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:32.906180 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-9069ecab-11e5-41be-bc67-dbb84830436b None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:32.908006 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-9069ecab-11e5-41be-bc67-dbb84830436b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:32.980929 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa449e-8bf9-40a9-8218-5270f4434cd0 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:33.051056 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f94f45-7c65-4714-b226-23442f593bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:33.051808 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f94f45-7c65-4714-b226-23442f593bfa None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:33.052159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f94f45-7c65-4714-b226-23442f593bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:33.052434 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f94f45-7c65-4714-b226-23442f593bfa None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:33.053165 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f94f45-7c65-4714-b226-23442f593bfa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:33.053476 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f94f45-7c65-4714-b226-23442f593bfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:33.055009 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f94f45-7c65-4714-b226-23442f593bfa None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:33.055787 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 84/163] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:20:33.071934 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 80/164] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:20:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:33.078229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069ecab-11e5-41be-bc67-dbb84830436b None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:33.172245 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:33.216974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069ecab-11e5-41be-bc67-dbb84830436b None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:33.256066 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df93205-f565-4d13-ad4b-2926710ea644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:33.256623 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df93205-f565-4d13-ad4b-2926710ea644 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:33.256951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df93205-f565-4d13-ad4b-2926710ea644 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:33.257228 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df93205-f565-4d13-ad4b-2926710ea644 None admin] PATH_INFO: `/v3/users/6d995078a79f4035bcbc04685a0579e2` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:33.262937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df93205-f565-4d13-ad4b-2926710ea644 None admin] RBAC: Authorizing `identity:get_user(user_id=6d995078a79f4035bcbc04685a0579e2)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:33.264454 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df93205-f565-4d13-ad4b-2926710ea644 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:33.266148 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 85/165] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:33 2022] GET /identity/v3/users/6d995078a79f4035bcbc04685a0579e2 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:33.272056 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f94f45-7c65-4714-b226-23442f593bfa None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:33.405159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f94f45-7c65-4714-b226-23442f593bfa None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:33.420821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df93205-f565-4d13-ad4b-2926710ea644 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:33.434611 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:33.474030 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170111b9-fecc-465d-bdc2-efc0390a87f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:33.474727 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170111b9-fecc-465d-bdc2-efc0390a87f7 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:33.475014 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170111b9-fecc-465d-bdc2-efc0390a87f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:33.475289 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170111b9-fecc-465d-bdc2-efc0390a87f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:33.562811 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df93205-f565-4d13-ad4b-2926710ea644 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:33.603277 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:33.614397 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f5474-02ea-48e7-9dfb-c45c3a870ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:33.615213 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f5474-02ea-48e7-9dfb-c45c3a870ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:33.615505 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f5474-02ea-48e7-9dfb-c45c3a870ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:33.615789 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f5474-02ea-48e7-9dfb-c45c3a870ac8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:33.631597 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f5474-02ea-48e7-9dfb-c45c3a870ac8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:33.643175 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-1e5f5474-02ea-48e7-9dfb-c45c3a870ac8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:20:33.643175 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:33.643175 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:33.643175 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:33.643175 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:33.643175 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:33.643175 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:33.643175 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:33.643175 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:33.643175 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:33.643175 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:33.643175 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:33.643175 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:33.643175 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:33.643175 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:33.643175 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:33.644023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:33.644023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:33.644023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:33.644023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:33.644023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:33.644023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:33.644023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:33.644023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:33.644023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:33.644023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:33.644023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:33.644023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:33.644023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:33.644023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:33.644023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:33.644023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:33.645020 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:33.645020 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:33.645020 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:33.645020 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:33.645020 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:33.645020 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:33.645020 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:33.645020 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:33.645020 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:33.645020 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:33.645020 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:20:33.645020 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:20:33.645020 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 86/166] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:20:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:33.645629 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f5474-02ea-48e7-9dfb-c45c3a870ac8 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:33.705692 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170111b9-fecc-465d-bdc2-efc0390a87f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:33.709183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170111b9-fecc-465d-bdc2-efc0390a87f7 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:33.875171 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f5474-02ea-48e7-9dfb-c45c3a870ac8 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:33.956598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad046604-9cde-4250-9fcd-b4cee264e302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:33.957250 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad046604-9cde-4250-9fcd-b4cee264e302 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:33.957536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad046604-9cde-4250-9fcd-b4cee264e302 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:33.957815 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad046604-9cde-4250-9fcd-b4cee264e302 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:33.966618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 81/167] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:20:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 695 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:20:33.967530 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170111b9-fecc-465d-bdc2-efc0390a87f7 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:33.989383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad046604-9cde-4250-9fcd-b4cee264e302 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:33.994916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-ad046604-9cde-4250-9fcd-b4cee264e302 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:20:33.994916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:33.994916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:33.994916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:33.994916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:33.994916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:33.994916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:33.994916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:33.994916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:33.994916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:33.994916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:33.994916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:33.994916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:33.994916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:33.994916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:33.994916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:33.995752 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:33.995752 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:33.995752 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:33.995752 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:33.995752 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:33.995752 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:33.995752 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:33.995752 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:33.995752 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:33.995752 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:33.995752 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:33.995752 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:33.995752 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:33.995752 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:33.995752 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:33.995752 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:33.996985 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:33.996985 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:33.996985 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:33.996985 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:33.996985 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:33.996985 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:33.996985 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:33.996985 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:33.996985 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:33.996985 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:33.996985 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:20:33.996985 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:20:33.996985 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 87/168] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:33.997601 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad046604-9cde-4250-9fcd-b4cee264e302 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:34.026642 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:34.119898 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:34.128669 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170111b9-fecc-465d-bdc2-efc0390a87f7 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:34.202161 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b9b01-d116-413c-a83d-0037387833f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:34.203262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b9b01-d116-413c-a83d-0037387833f1 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:34.203824 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b9b01-d116-413c-a83d-0037387833f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:34.204310 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b9b01-d116-413c-a83d-0037387833f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:34.205186 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b9b01-d116-413c-a83d-0037387833f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:34.205681 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b9b01-d116-413c-a83d-0037387833f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:34.207475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b9b01-d116-413c-a83d-0037387833f1 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:34.213839 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad046604-9cde-4250-9fcd-b4cee264e302 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:34.223130 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 82/169] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:34.224439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b9b01-d116-413c-a83d-0037387833f1 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:34.254049 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70dc813-7340-4180-8240-b8c383ab2a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:34.254732 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70dc813-7340-4180-8240-b8c383ab2a96 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:34.255013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70dc813-7340-4180-8240-b8c383ab2a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:34.255288 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70dc813-7340-4180-8240-b8c383ab2a96 None admin] PATH_INFO: `/v3/projects/a3f472f7ed974397b7020b42b4f22bd3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:34.256847 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70dc813-7340-4180-8240-b8c383ab2a96 None admin] RBAC: Authorizing `identity:get_project(project_id=a3f472f7ed974397b7020b42b4f22bd3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:34.258509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70dc813-7340-4180-8240-b8c383ab2a96 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:34.260729 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 88/170] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:33 2022] GET /identity/v3/projects/a3f472f7ed974397b7020b42b4f22bd3 => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:34.261316 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70dc813-7340-4180-8240-b8c383ab2a96 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:34.397393 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:34.399743 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70dc813-7340-4180-8240-b8c383ab2a96 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:34.433183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b9b01-d116-413c-a83d-0037387833f1 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:34.439354 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:34.462976 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305468d8-8cdd-49f6-8292-487de061fa4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:34.463617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305468d8-8cdd-49f6-8292-487de061fa4a None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:34.464086 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305468d8-8cdd-49f6-8292-487de061fa4a None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:34.464360 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305468d8-8cdd-49f6-8292-487de061fa4a None admin] PATH_INFO: `/v3/users/eed887c1c93d4f24959b0f90e215c424` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:34.465749 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305468d8-8cdd-49f6-8292-487de061fa4a None admin] RBAC: Authorizing `identity:get_user(user_id=eed887c1c93d4f24959b0f90e215c424)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:34.467893 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305468d8-8cdd-49f6-8292-487de061fa4a None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:34.469636 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 89/171] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:34 2022] GET /identity/v3/users/eed887c1c93d4f24959b0f90e215c424 => generated 247 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:34.470545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305468d8-8cdd-49f6-8292-487de061fa4a None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:34.540997 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe369e5-2ce9-4ab9-9e89-3062ab3de4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:34.541619 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe369e5-2ce9-4ab9-9e89-3062ab3de4f9 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:34.541902 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe369e5-2ce9-4ab9-9e89-3062ab3de4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:34.542173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe369e5-2ce9-4ab9-9e89-3062ab3de4f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:34.542871 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe369e5-2ce9-4ab9-9e89-3062ab3de4f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:34.543170 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe369e5-2ce9-4ab9-9e89-3062ab3de4f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:34.544547 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe369e5-2ce9-4ab9-9e89-3062ab3de4f9 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:34.562400 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 83/172] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 1 16:20:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:34.563665 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ea83c-4580-4cab-b8b7-004036fa0cd7 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:34.563952 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ea83c-4580-4cab-b8b7-004036fa0cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:34.564223 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ea83c-4580-4cab-b8b7-004036fa0cd7 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:34.565650 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 84/173] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:34.566184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ea83c-4580-4cab-b8b7-004036fa0cd7 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:34.583579 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305468d8-8cdd-49f6-8292-487de061fa4a None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:34.619712 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04426fe9-d6ee-4f5f-9189-bf0e8f0e473b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:34.620332 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04426fe9-d6ee-4f5f-9189-bf0e8f0e473b None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:34.620622 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04426fe9-d6ee-4f5f-9189-bf0e8f0e473b None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:34.620898 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04426fe9-d6ee-4f5f-9189-bf0e8f0e473b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:34.621626 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04426fe9-d6ee-4f5f-9189-bf0e8f0e473b None admin] RBAC: Adding query filter params (role.id=f2a6e936aa294323ba7087bf377276b2, scope.project.id=a3f472f7ed974397b7020b42b4f22bd3, user.id=6d995078a79f4035bcbc04685a0579e2) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:34.621934 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04426fe9-d6ee-4f5f-9189-bf0e8f0e473b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:34.623413 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04426fe9-d6ee-4f5f-9189-bf0e8f0e473b None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:34.629863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 90/174] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:20:34 2022] GET /identity/v3/role_assignments?user.id=6d995078a79f4035bcbc04685a0579e2&scope.project.id=a3f472f7ed974397b7020b42b4f22bd3&role.id=f2a6e936aa294323ba7087bf377276b2 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:34.631134 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdcb13-abd1-41f0-bf7e-c84181aa88d6 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:34.631431 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdcb13-abd1-41f0-bf7e-c84181aa88d6 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:34.631710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdcb13-abd1-41f0-bf7e-c84181aa88d6 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:34.632927 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 91/175] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:34.633620 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bdcb13-abd1-41f0-bf7e-c84181aa88d6 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:34.744543 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:34.754266 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ea83c-4580-4cab-b8b7-004036fa0cd7 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:34.818266 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c657f-2a3c-4ea7-ac1d-50e497810070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:34.818889 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c657f-2a3c-4ea7-ac1d-50e497810070 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:34.819180 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c657f-2a3c-4ea7-ac1d-50e497810070 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:34.819454 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c657f-2a3c-4ea7-ac1d-50e497810070 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:34.833375 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:34.834642 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bdcb13-abd1-41f0-bf7e-c84181aa88d6 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:34.892368 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1bab0c-e389-4a51-8089-85f24942991f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:34.892735 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bab0c-e389-4a51-8089-85f24942991f None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:34.893185 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bab0c-e389-4a51-8089-85f24942991f None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:34.893598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bab0c-e389-4a51-8089-85f24942991f None admin] PATH_INFO: `/v3/users/6d995078a79f4035bcbc04685a0579e2` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:34.894931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1bab0c-e389-4a51-8089-85f24942991f None admin] RBAC: Authorizing `identity:get_user(user_id=6d995078a79f4035bcbc04685a0579e2)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:34.896531 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1bab0c-e389-4a51-8089-85f24942991f None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:34.898330 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 92/176] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:34 2022] GET /identity/v3/users/6d995078a79f4035bcbc04685a0579e2 => generated 275 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:34.899630 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d00f1-31b1-40e9-a964-adaa31952ec5 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:34.899946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d00f1-31b1-40e9-a964-adaa31952ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:34.900223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d00f1-31b1-40e9-a964-adaa31952ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:34.919594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c657f-2a3c-4ea7-ac1d-50e497810070 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:34.921533 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c657f-2a3c-4ea7-ac1d-50e497810070 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:34.923012 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-959d00f1-31b1-40e9-a964-adaa31952ec5 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:34.924183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-959d00f1-31b1-40e9-a964-adaa31952ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:34.994181 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 93/177] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:35.003623 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659da45-7442-4d5c-aad9-ea4ac78c4a0e None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:35.003872 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659da45-7442-4d5c-aad9-ea4ac78c4a0e None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:35.004107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659da45-7442-4d5c-aad9-ea4ac78c4a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:35.023105 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-8659da45-7442-4d5c-aad9-ea4ac78c4a0e None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:35.024362 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-8659da45-7442-4d5c-aad9-ea4ac78c4a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:35.033461 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 85/178] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:20:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:20:35.035707 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a56d60-efb4-43b0-bd6e-a17c7a28a815 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:35.036462 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a56d60-efb4-43b0-bd6e-a17c7a28a815 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:35.036933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a56d60-efb4-43b0-bd6e-a17c7a28a815 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:35.038628 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 86/179] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:35.043871 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a56d60-efb4-43b0-bd6e-a17c7a28a815 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:35.075423 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:35.089280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:35.114277 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 94/180] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:35.114277 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8659da45-7442-4d5c-aad9-ea4ac78c4a0e None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:35.167764 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a56d60-efb4-43b0-bd6e-a17c7a28a815 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:35.206405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40bbe59-0f1c-4527-ae82-ffdbc5cbad24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:35.207022 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40bbe59-0f1c-4527-ae82-ffdbc5cbad24 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:35.207305 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40bbe59-0f1c-4527-ae82-ffdbc5cbad24 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:35.207582 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40bbe59-0f1c-4527-ae82-ffdbc5cbad24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:35.227745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8659da45-7442-4d5c-aad9-ea4ac78c4a0e None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:35.265397 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d64b8-c71d-48a5-b285-fd6e6a5a794c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:35.265957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d64b8-c71d-48a5-b285-fd6e6a5a794c None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:35.266255 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d64b8-c71d-48a5-b285-fd6e6a5a794c None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:35.266525 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d64b8-c71d-48a5-b285-fd6e6a5a794c None admin] PATH_INFO: `/v3/projects/a3f472f7ed974397b7020b42b4f22bd3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:35.267729 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223d64b8-c71d-48a5-b285-fd6e6a5a794c None admin] RBAC: Authorizing `identity:get_project(project_id=a3f472f7ed974397b7020b42b4f22bd3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:35.269300 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223d64b8-c71d-48a5-b285-fd6e6a5a794c None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:35.271253 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 95/181] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:35 2022] GET /identity/v3/projects/a3f472f7ed974397b7020b42b4f22bd3 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:35.272418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d31ee-a459-46ad-9613-27a5db01cba1 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:35.272740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d31ee-a459-46ad-9613-27a5db01cba1 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:35.273082 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d31ee-a459-46ad-9613-27a5db01cba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:35.277980 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40bbe59-0f1c-4527-ae82-ffdbc5cbad24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:35.279498 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40bbe59-0f1c-4527-ae82-ffdbc5cbad24 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:35.292768 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-0d8d31ee-a459-46ad-9613-27a5db01cba1 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:35.294065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-0d8d31ee-a459-46ad-9613-27a5db01cba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:35.323464 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:35.341892 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:35.371705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 96/182] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:35.378487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e743b8-366e-471b-acb1-7e2e38432d26 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:35.378487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e743b8-366e-471b-acb1-7e2e38432d26 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:35.378487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e743b8-366e-471b-acb1-7e2e38432d26 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:35.378487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 97/183] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:35.378487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e743b8-366e-471b-acb1-7e2e38432d26 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:35.399609 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 87/184] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:20:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:20:35.400457 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40bbe59-0f1c-4527-ae82-ffdbc5cbad24 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:35.493875 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e743b8-366e-471b-acb1-7e2e38432d26 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:35.500798 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40bbe59-0f1c-4527-ae82-ffdbc5cbad24 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:35.528675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5dd8bd-60b6-4d99-94bd-58c41e3c0a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:35.529320 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5dd8bd-60b6-4d99-94bd-58c41e3c0a11 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:35.529600 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5dd8bd-60b6-4d99-94bd-58c41e3c0a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:35.530503 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5dd8bd-60b6-4d99-94bd-58c41e3c0a11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:35.531234 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5dd8bd-60b6-4d99-94bd-58c41e3c0a11 None admin] RBAC: Adding query filter params (role.id=e8ba063905484d22993bcf04830091a3, scope.project.id=a3f472f7ed974397b7020b42b4f22bd3, user.id=eed887c1c93d4f24959b0f90e215c424) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:35.531542 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5dd8bd-60b6-4d99-94bd-58c41e3c0a11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:35.532985 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5dd8bd-60b6-4d99-94bd-58c41e3c0a11 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:35.535490 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b6322-717d-4c88-bc19-52d09381cb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:35.536124 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b6322-717d-4c88-bc19-52d09381cb0f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:35.536408 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b6322-717d-4c88-bc19-52d09381cb0f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:35.536684 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b6322-717d-4c88-bc19-52d09381cb0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:35.539491 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 98/185] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:20:35 2022] GET /identity/v3/role_assignments?user.id=eed887c1c93d4f24959b0f90e215c424&scope.project.id=a3f472f7ed974397b7020b42b4f22bd3&role.id=e8ba063905484d22993bcf04830091a3 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:35.540692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc707367-8bdb-4987-a310-f6d8998c02ee None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:35.540981 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc707367-8bdb-4987-a310-f6d8998c02ee None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:35.541409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc707367-8bdb-4987-a310-f6d8998c02ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:35.557555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b6322-717d-4c88-bc19-52d09381cb0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:35.563401 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-0d3b6322-717d-4c88-bc19-52d09381cb0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:20:35.563401 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:35.563401 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:35.563401 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:35.563401 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:35.563401 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:35.563401 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:35.563401 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:35.563401 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:35.563401 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:35.563401 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:35.563401 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:35.563401 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:35.563401 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:35.563401 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:35.563401 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:35.564406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:35.564406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:35.564406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:35.564406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:35.564406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:35.564406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:35.564406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:35.564406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:35.564406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:35.564406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:35.564406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:35.564406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:35.564406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:35.564406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:35.564406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:35.564406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:35.565184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:35.565184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:35.565184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:35.565184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:35.565184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:35.565184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:35.565184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:35.565184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:35.565184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:35.565184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:35.565184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:20:35.565184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:20:35.565184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 88/186] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:20:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:35.565868 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b6322-717d-4c88-bc19-52d09381cb0f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:35.569501 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:35.571164 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-dc707367-8bdb-4987-a310-f6d8998c02ee None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:35.573069 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-dc707367-8bdb-4987-a310-f6d8998c02ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:35.613888 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:35.656849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 99/187] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:35.656849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc707367-8bdb-4987-a310-f6d8998c02ee None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:35.681173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b6322-717d-4c88-bc19-52d09381cb0f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:35.723303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e92b0-e86d-4aa8-bd70-61d38a519a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:35.723990 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e92b0-e86d-4aa8-bd70-61d38a519a33 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:35.724275 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e92b0-e86d-4aa8-bd70-61d38a519a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:35.724545 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e92b0-e86d-4aa8-bd70-61d38a519a33 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:35.736362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1e92b0-e86d-4aa8-bd70-61d38a519a33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:35.741190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-fb1e92b0-e86d-4aa8-bd70-61d38a519a33 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:20:35.741190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:35.741190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:35.741190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:35.741190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:35.741190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:35.741190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:35.741190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:35.741190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:35.741190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:35.741190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:35.741190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:35.741190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:35.741190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:35.741190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:35.741190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:35.742044 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:35.742044 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:35.742044 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:35.742044 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:35.742044 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:35.742044 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:35.742044 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:35.742044 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:35.742044 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:35.742044 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:35.742044 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:35.742044 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:35.742044 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:35.742044 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:35.742044 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:35.742044 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:35.742986 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:35.742986 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:35.742986 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:35.742986 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:35.742986 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:35.742986 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:35.742986 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:35.742986 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:35.742986 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:35.742986 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:35.742986 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:20:35.742986 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:20:35.742986 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 89/188] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 1 16:20:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:35.743648 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e92b0-e86d-4aa8-bd70-61d38a519a33 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:35.785527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc707367-8bdb-4987-a310-f6d8998c02ee None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:35.818528 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:35.827253 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a44e4-b549-4146-906a-18f61fd30e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:35.827898 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a44e4-b549-4146-906a-18f61fd30e6e None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:35.828165 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a44e4-b549-4146-906a-18f61fd30e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:35.832229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a44e4-b549-4146-906a-18f61fd30e6e None admin] PATH_INFO: `/v3/projects/1cf23dd85509483f9d433c73e9329b89` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:35.832229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7a44e4-b549-4146-906a-18f61fd30e6e None admin] RBAC: Authorizing `identity:get_project(project_id=1cf23dd85509483f9d433c73e9329b89)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:35.833358 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7a44e4-b549-4146-906a-18f61fd30e6e None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:35.835276 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 100/189] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:35 2022] GET /identity/v3/projects/1cf23dd85509483f9d433c73e9329b89 => generated 312 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:35.835831 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a44e4-b549-4146-906a-18f61fd30e6e None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:35.870530 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e92b0-e86d-4aa8-bd70-61d38a519a33 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:35.876591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:35.911086 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97b946-c134-4ed6-bf2e-d5cb53ce645f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:35.911958 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97b946-c134-4ed6-bf2e-d5cb53ce645f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:35.912408 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97b946-c134-4ed6-bf2e-d5cb53ce645f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:35.912840 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97b946-c134-4ed6-bf2e-d5cb53ce645f None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:35.913656 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e97b946-c134-4ed6-bf2e-d5cb53ce645f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:35.914132 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e97b946-c134-4ed6-bf2e-d5cb53ce645f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:35.915600 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e97b946-c134-4ed6-bf2e-d5cb53ce645f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:35.923863 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 90/190] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:35.924669 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97b946-c134-4ed6-bf2e-d5cb53ce645f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:35.942853 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a44e4-b549-4146-906a-18f61fd30e6e None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:35.976353 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14513e2d-5639-4984-9e5d-95e0a2d482cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:35.976980 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14513e2d-5639-4984-9e5d-95e0a2d482cd None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:35.977260 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14513e2d-5639-4984-9e5d-95e0a2d482cd None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:35.977529 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14513e2d-5639-4984-9e5d-95e0a2d482cd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:35.978659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14513e2d-5639-4984-9e5d-95e0a2d482cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:35.980129 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14513e2d-5639-4984-9e5d-95e0a2d482cd None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:35.981711 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 101/191] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:20:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:35.982563 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14513e2d-5639-4984-9e5d-95e0a2d482cd None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:36.023909 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97b946-c134-4ed6-bf2e-d5cb53ce645f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:36.046424 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:36.057371 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b32fe-1364-4d94-a567-3f4f586c1bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:36.058004 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b32fe-1364-4d94-a567-3f4f586c1bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:36.058326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b32fe-1364-4d94-a567-3f4f586c1bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:36.058596 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b32fe-1364-4d94-a567-3f4f586c1bf5 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:36.059257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b32fe-1364-4d94-a567-3f4f586c1bf5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:36.059558 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b32fe-1364-4d94-a567-3f4f586c1bf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:36.060911 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b32fe-1364-4d94-a567-3f4f586c1bf5 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:36.069277 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 91/192] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:20:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:36.070130 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b32fe-1364-4d94-a567-3f4f586c1bf5 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:36.087464 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14513e2d-5639-4984-9e5d-95e0a2d482cd None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:36.094931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:36.121675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68cecd4-73f0-49b1-8141-c9f21cf090dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:36.122416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cecd4-73f0-49b1-8141-c9f21cf090dc None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:36.122701 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cecd4-73f0-49b1-8141-c9f21cf090dc None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:36.122972 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cecd4-73f0-49b1-8141-c9f21cf090dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:36.124488 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68cecd4-73f0-49b1-8141-c9f21cf090dc None admin] RBAC: Adding query filter params (role.id=45545334e1fc473c9d85fc91a9b0a35c, scope.project.id=1cf23dd85509483f9d433c73e9329b89, user.id=6d995078a79f4035bcbc04685a0579e2) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:36.124822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68cecd4-73f0-49b1-8141-c9f21cf090dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:36.126284 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68cecd4-73f0-49b1-8141-c9f21cf090dc None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:36.131927 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 102/193] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:20:35 2022] GET /identity/v3/role_assignments?user.id=6d995078a79f4035bcbc04685a0579e2&scope.project.id=1cf23dd85509483f9d433c73e9329b89&role.id=45545334e1fc473c9d85fc91a9b0a35c => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:36.135095 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68cecd4-73f0-49b1-8141-c9f21cf090dc None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:36.173784 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b32fe-1364-4d94-a567-3f4f586c1bf5 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:36.213771 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05057b9e-76cc-4b51-8b91-3b0151fadb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:36.214452 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05057b9e-76cc-4b51-8b91-3b0151fadb94 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:36.214738 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05057b9e-76cc-4b51-8b91-3b0151fadb94 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:36.215013 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05057b9e-76cc-4b51-8b91-3b0151fadb94 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:36.229331 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05057b9e-76cc-4b51-8b91-3b0151fadb94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:36.235594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-05057b9e-76cc-4b51-8b91-3b0151fadb94 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:20:36.235594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:36.235594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:36.235594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:36.235594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:36.235594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:36.235594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:36.235594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:36.235594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:36.235594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:36.235594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:36.235594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:36.235594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:36.235594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:36.235594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:36.235594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:36.236527 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:36.236527 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:36.236527 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:36.236527 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:36.236527 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:36.236527 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:36.236527 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:36.236527 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:36.236527 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:36.236527 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:36.236527 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:36.236527 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:36.236527 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:36.236527 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:36.236527 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:36.236527 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:36.238790 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:36.238790 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:36.238790 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:36.238790 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:36.238790 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:36.238790 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:36.238790 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:36.238790 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:36.238790 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:36.238790 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:36.238790 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:20:36.238790 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:20:36.238790 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 92/194] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:36.238790 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05057b9e-76cc-4b51-8b91-3b0151fadb94 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:36.253206 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68cecd4-73f0-49b1-8141-c9f21cf090dc None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:36.292689 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:36.294361 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b9516f-303d-4a66-8fbd-29e1480bfbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:36.295023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9516f-303d-4a66-8fbd-29e1480bfbaf None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:36.295311 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9516f-303d-4a66-8fbd-29e1480bfbaf None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:36.295638 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9516f-303d-4a66-8fbd-29e1480bfbaf None admin] PATH_INFO: `/v3/users/08bdf9a0060d472fa267472d8bbe5585` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:36.310802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b9516f-303d-4a66-8fbd-29e1480bfbaf None admin] RBAC: Authorizing `identity:get_user(user_id=08bdf9a0060d472fa267472d8bbe5585)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:36.312942 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b9516f-303d-4a66-8fbd-29e1480bfbaf None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:36.314764 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 103/195] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:36 2022] GET /identity/v3/users/08bdf9a0060d472fa267472d8bbe5585 => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:36.318612 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b9516f-303d-4a66-8fbd-29e1480bfbaf None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:36.357330 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05057b9e-76cc-4b51-8b91-3b0151fadb94 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:36.358603 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:36.402363 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee788180-67a7-4831-baf9-bac8e4d0680a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:36.402989 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee788180-67a7-4831-baf9-bac8e4d0680a None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:36.403270 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee788180-67a7-4831-baf9-bac8e4d0680a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:36.403541 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee788180-67a7-4831-baf9-bac8e4d0680a None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:36.404191 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee788180-67a7-4831-baf9-bac8e4d0680a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:36.405673 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee788180-67a7-4831-baf9-bac8e4d0680a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:36.426576 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 93/196] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 1 16:20:36 2022] POST /identity/v3/users => generated 297 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:20:36.427389 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee788180-67a7-4831-baf9-bac8e4d0680a None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:36.436243 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b9516f-303d-4a66-8fbd-29e1480bfbaf None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:36.474783 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7cbec4-794a-419a-ad8a-8092b890fb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:36.475459 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cbec4-794a-419a-ad8a-8092b890fb4e None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:36.475745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cbec4-794a-419a-ad8a-8092b890fb4e None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:36.476022 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cbec4-794a-419a-ad8a-8092b890fb4e None admin] PATH_INFO: `/v3/users/08bdf9a0060d472fa267472d8bbe5585` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:36.477250 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7cbec4-794a-419a-ad8a-8092b890fb4e None admin] RBAC: Authorizing `identity:get_user(user_id=08bdf9a0060d472fa267472d8bbe5585)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:36.478801 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7cbec4-794a-419a-ad8a-8092b890fb4e None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:36.480744 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 104/197] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:36 2022] GET /identity/v3/users/08bdf9a0060d472fa267472d8bbe5585 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:36.481354 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7cbec4-794a-419a-ad8a-8092b890fb4e None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:36.548374 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee788180-67a7-4831-baf9-bac8e4d0680a None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:36.550171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:36.593188 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:36.593358 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e19ee5d-e20c-4028-b850-70c44d6184ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:36.593918 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19ee5d-e20c-4028-b850-70c44d6184ba None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:36.594288 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19ee5d-e20c-4028-b850-70c44d6184ba None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:36.594731 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19ee5d-e20c-4028-b850-70c44d6184ba None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:36.595729 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e19ee5d-e20c-4028-b850-70c44d6184ba None admin] RBAC: Adding query filter params (name=reader) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:36.596132 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e19ee5d-e20c-4028-b850-70c44d6184ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:36.598554 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e19ee5d-e20c-4028-b850-70c44d6184ba None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:36.601337 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7cbec4-794a-419a-ad8a-8092b890fb4e None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:36.607097 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 94/198] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 1 16:20:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:36.608567 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e19ee5d-e20c-4028-b850-70c44d6184ba None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:36.645626 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde8a90-df54-4a4a-8150-d20ef894b7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:36.646259 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde8a90-df54-4a4a-8150-d20ef894b7d3 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:36.646609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde8a90-df54-4a4a-8150-d20ef894b7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:36.646882 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde8a90-df54-4a4a-8150-d20ef894b7d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:36.732412 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cde8a90-df54-4a4a-8150-d20ef894b7d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:36.734767 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cde8a90-df54-4a4a-8150-d20ef894b7d3 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:36.736051 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e19ee5d-e20c-4028-b850-70c44d6184ba None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:36.775480 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f141a512-cdd7-45ee-b031-5909c18e6ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:36.776304 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141a512-cdd7-45ee-b031-5909c18e6ef6 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:36.776755 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141a512-cdd7-45ee-b031-5909c18e6ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:36.777100 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141a512-cdd7-45ee-b031-5909c18e6ef6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:36.819049 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:36.849023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:36.856407 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 105/199] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:20:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:20:36.857342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde8a90-df54-4a4a-8150-d20ef894b7d3 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:36.876113 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f141a512-cdd7-45ee-b031-5909c18e6ef6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:36.878449 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f141a512-cdd7-45ee-b031-5909c18e6ef6 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:36.979612 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde8a90-df54-4a4a-8150-d20ef894b7d3 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:36.999067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 95/200] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:20:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:20:36.999977 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f141a512-cdd7-45ee-b031-5909c18e6ef6 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:37.024866 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8463d-eda7-4262-b70c-9ad511c18c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:37.025768 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8463d-eda7-4262-b70c-9ad511c18c2e None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:37.026058 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8463d-eda7-4262-b70c-9ad511c18c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:37.026797 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8463d-eda7-4262-b70c-9ad511c18c2e None admin] PATH_INFO: `/v3/users/4955a8d9be064e8c8bddf4d21a8964f5` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:37.028183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd8463d-eda7-4262-b70c-9ad511c18c2e None admin] RBAC: Authorizing `identity:get_user(user_id=4955a8d9be064e8c8bddf4d21a8964f5)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:37.030897 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd8463d-eda7-4262-b70c-9ad511c18c2e None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:37.032577 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 106/201] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:36 2022] GET /identity/v3/users/4955a8d9be064e8c8bddf4d21a8964f5 => generated 289 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:37.034013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8463d-eda7-4262-b70c-9ad511c18c2e None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:37.077051 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:37.118625 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f141a512-cdd7-45ee-b031-5909c18e6ef6 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:37.119660 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:37.155900 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c1bf7-fe1a-4e68-8d31-c2d36323c939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:37.156253 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c1bf7-fe1a-4e68-8d31-c2d36323c939 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:37.156525 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c1bf7-fe1a-4e68-8d31-c2d36323c939 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:37.156802 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c1bf7-fe1a-4e68-8d31-c2d36323c939 None admin] PATH_INFO: `/v3/projects/cc5b99bf5631484eb25353883854e856` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:37.157941 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c1bf7-fe1a-4e68-8d31-c2d36323c939 None admin] RBAC: Authorizing `identity:get_project(project_id=cc5b99bf5631484eb25353883854e856)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:37.159488 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c1bf7-fe1a-4e68-8d31-c2d36323c939 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:37.161377 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 96/202] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:36 2022] GET /identity/v3/projects/cc5b99bf5631484eb25353883854e856 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:37.162136 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c1bf7-fe1a-4e68-8d31-c2d36323c939 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:37.171346 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8463d-eda7-4262-b70c-9ad511c18c2e None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:37.215331 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1162dd-1972-437f-b0ec-2c32721a27b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:37.216726 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1162dd-1972-437f-b0ec-2c32721a27b8 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:37.218152 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1162dd-1972-437f-b0ec-2c32721a27b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:37.218657 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1162dd-1972-437f-b0ec-2c32721a27b8 None admin] PATH_INFO: `/v3/projects/cc5b99bf5631484eb25353883854e856` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:37.220471 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1162dd-1972-437f-b0ec-2c32721a27b8 None admin] RBAC: Authorizing `identity:get_project(project_id=cc5b99bf5631484eb25353883854e856)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:37.222581 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1162dd-1972-437f-b0ec-2c32721a27b8 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:37.225185 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 107/203] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:37 2022] GET /identity/v3/projects/cc5b99bf5631484eb25353883854e856 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:37.226569 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1162dd-1972-437f-b0ec-2c32721a27b8 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:37.283249 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c1bf7-fe1a-4e68-8d31-c2d36323c939 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:37.318778 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:37.324128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b50d97-88cf-41dd-88f8-d1a0f9087e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:37.324829 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b50d97-88cf-41dd-88f8-d1a0f9087e2b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:37.325726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b50d97-88cf-41dd-88f8-d1a0f9087e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:37.325843 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b50d97-88cf-41dd-88f8-d1a0f9087e2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:37.350796 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1162dd-1972-437f-b0ec-2c32721a27b8 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:37.391355 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:37.398751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a00ea52-c88c-4787-bba3-99396d574f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:37.399458 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b50d97-88cf-41dd-88f8-d1a0f9087e2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:37.399730 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00ea52-c88c-4787-bba3-99396d574f9f None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:37.400160 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00ea52-c88c-4787-bba3-99396d574f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:37.400573 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00ea52-c88c-4787-bba3-99396d574f9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:37.401335 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b50d97-88cf-41dd-88f8-d1a0f9087e2b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:37.401625 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a00ea52-c88c-4787-bba3-99396d574f9f None admin] RBAC: Adding query filter params (role.id=e8ba063905484d22993bcf04830091a3, scope.project.id=cc5b99bf5631484eb25353883854e856, user.id=08bdf9a0060d472fa267472d8bbe5585) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:37.402077 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a00ea52-c88c-4787-bba3-99396d574f9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:37.403923 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a00ea52-c88c-4787-bba3-99396d574f9f None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:37.411019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 108/204] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:20:37 2022] GET /identity/v3/role_assignments?user.id=08bdf9a0060d472fa267472d8bbe5585&scope.project.id=cc5b99bf5631484eb25353883854e856&role.id=e8ba063905484d22993bcf04830091a3 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:37.411967 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a00ea52-c88c-4787-bba3-99396d574f9f None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:37.526153 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 97/205] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:20:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:20:37.533942 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b50d97-88cf-41dd-88f8-d1a0f9087e2b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:37.571679 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:37.622646 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a00ea52-c88c-4787-bba3-99396d574f9f None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:37.660890 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b50d97-88cf-41dd-88f8-d1a0f9087e2b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:37.675549 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbe2c8-e0aa-42d7-a13e-bd6427c6332b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:37.676158 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbe2c8-e0aa-42d7-a13e-bd6427c6332b None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:37.676444 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbe2c8-e0aa-42d7-a13e-bd6427c6332b None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:37.676716 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbe2c8-e0aa-42d7-a13e-bd6427c6332b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:37.677450 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bbe2c8-e0aa-42d7-a13e-bd6427c6332b None admin] RBAC: Adding query filter params (role.id=f2a6e936aa294323ba7087bf377276b2, scope.project.id=cc5b99bf5631484eb25353883854e856, user.id=08bdf9a0060d472fa267472d8bbe5585) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:37.677772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bbe2c8-e0aa-42d7-a13e-bd6427c6332b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:37.679251 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bbe2c8-e0aa-42d7-a13e-bd6427c6332b None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:37.684958 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 109/206] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:20:37 2022] GET /identity/v3/role_assignments?user.id=08bdf9a0060d472fa267472d8bbe5585&scope.project.id=cc5b99bf5631484eb25353883854e856&role.id=f2a6e936aa294323ba7087bf377276b2 => generated 259 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:37.703729 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502aee0-0be7-4642-a24b-91d137cc1a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:37.704751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502aee0-0be7-4642-a24b-91d137cc1a5d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:37.705237 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502aee0-0be7-4642-a24b-91d137cc1a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:37.705695 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502aee0-0be7-4642-a24b-91d137cc1a5d None admin] PATH_INFO: `/v3/projects/a3f472f7ed974397b7020b42b4f22bd3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:37.707368 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d502aee0-0be7-4642-a24b-91d137cc1a5d None admin] RBAC: Authorizing `identity:get_project(project_id=a3f472f7ed974397b7020b42b4f22bd3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:37.709132 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d502aee0-0be7-4642-a24b-91d137cc1a5d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:37.711587 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 98/207] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:37 2022] GET /identity/v3/projects/a3f472f7ed974397b7020b42b4f22bd3 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:20:37.718672 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbe2c8-e0aa-42d7-a13e-bd6427c6332b None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:37.790915 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:37.835251 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbe2c8-e0aa-42d7-a13e-bd6427c6332b None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:37.873102 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2f72c-5f4f-44c7-a512-ee6317b16a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:37.874321 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2f72c-5f4f-44c7-a512-ee6317b16a5c None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:37.874805 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2f72c-5f4f-44c7-a512-ee6317b16a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:37.875261 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2f72c-5f4f-44c7-a512-ee6317b16a5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:37.876172 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b2f72c-5f4f-44c7-a512-ee6317b16a5c None admin] RBAC: Adding query filter params (role.id=12fe0aa7f1174edfb109315b88d6267b, scope.project.id=a3f472f7ed974397b7020b42b4f22bd3, user.id=4955a8d9be064e8c8bddf4d21a8964f5) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:37.876655 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b2f72c-5f4f-44c7-a512-ee6317b16a5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:37.878290 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b2f72c-5f4f-44c7-a512-ee6317b16a5c None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:37.884579 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 110/208] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:20:37 2022] GET /identity/v3/role_assignments?user.id=4955a8d9be064e8c8bddf4d21a8964f5&scope.project.id=a3f472f7ed974397b7020b42b4f22bd3&role.id=12fe0aa7f1174edfb109315b88d6267b => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:38.662346 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6c3f4-3219-4311-9585-edbdb1b24369 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:38.663340 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6c3f4-3219-4311-9585-edbdb1b24369 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:38.663825 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6c3f4-3219-4311-9585-edbdb1b24369 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:38.665609 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 99/209] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:38.671624 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d10c7a-8d87-4b69-b51d-e8555e704729 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:38.672420 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d10c7a-8d87-4b69-b51d-e8555e704729 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:38.672910 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d10c7a-8d87-4b69-b51d-e8555e704729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:38.692634 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-10d10c7a-8d87-4b69-b51d-e8555e704729 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:38.694147 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-10d10c7a-8d87-4b69-b51d-e8555e704729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:38.736649 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794765f9-5c61-4d70-8e0e-9b8782d0054f None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:38.736955 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794765f9-5c61-4d70-8e0e-9b8782d0054f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:38.737206 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794765f9-5c61-4d70-8e0e-9b8782d0054f None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:38.738701 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 100/210] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:38.745334 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca0440-f823-42ee-b6e3-9701fd4ec212 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:38.745710 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca0440-f823-42ee-b6e3-9701fd4ec212 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:38.746061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca0440-f823-42ee-b6e3-9701fd4ec212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:38.765303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-d6ca0440-f823-42ee-b6e3-9701fd4ec212 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:38.766605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d6ca0440-f823-42ee-b6e3-9701fd4ec212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:38.772905 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 111/211] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:20:38.843199 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 101/212] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:20:39.065095 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d10c7a-8d87-4b69-b51d-e8555e704729 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:39.113570 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca0440-f823-42ee-b6e3-9701fd4ec212 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:39.127959 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:39.157830 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:39.177729 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d10c7a-8d87-4b69-b51d-e8555e704729 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:39.212792 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690e0cb-ece1-4361-9984-f4cd41e5124c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:39.213440 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690e0cb-ece1-4361-9984-f4cd41e5124c None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:39.213734 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690e0cb-ece1-4361-9984-f4cd41e5124c None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:39.213983 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690e0cb-ece1-4361-9984-f4cd41e5124c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:39.225473 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4690e0cb-ece1-4361-9984-f4cd41e5124c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:39.230552 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-4690e0cb-ece1-4361-9984-f4cd41e5124c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:20:39.230552 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:39.230552 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:39.230552 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:39.230552 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:39.230552 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:39.230552 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:39.230552 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:39.230552 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:39.230552 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:39.230552 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:39.230552 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:39.230552 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:39.230552 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:39.230552 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:39.230552 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:39.231393 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:39.231393 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:39.231393 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:39.231393 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:39.231393 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:39.231393 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:39.231393 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:39.231393 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:39.231393 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:39.231393 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:39.231393 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:39.231393 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:39.231393 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:39.231393 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:39.231393 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:39.231393 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:39.237493 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:39.237493 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:39.237493 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:39.237493 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:39.237493 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:39.237493 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:39.237493 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:39.237493 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:39.237493 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:39.237493 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:39.237493 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:20:39.237493 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:20:39.237493 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 112/213] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:39.240643 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690e0cb-ece1-4361-9984-f4cd41e5124c None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:39.258687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca0440-f823-42ee-b6e3-9701fd4ec212 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:39.304068 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d23c6-4cb6-481c-8c13-349df6a3c938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:39.305287 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d23c6-4cb6-481c-8c13-349df6a3c938 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:39.306164 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d23c6-4cb6-481c-8c13-349df6a3c938 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:39.306636 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d23c6-4cb6-481c-8c13-349df6a3c938 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:39.308229 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d23c6-4cb6-481c-8c13-349df6a3c938 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:39.310706 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d23c6-4cb6-481c-8c13-349df6a3c938 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:39.313177 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 102/214] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:20:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:20:39.318959 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d23c6-4cb6-481c-8c13-349df6a3c938 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:39.367034 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690e0cb-ece1-4361-9984-f4cd41e5124c None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:39.382063 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:39.404520 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac0bb9-54b1-456f-92ac-f645b1e4e3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:39.406086 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac0bb9-54b1-456f-92ac-f645b1e4e3aa None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:39.406086 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac0bb9-54b1-456f-92ac-f645b1e4e3aa None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:39.406416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac0bb9-54b1-456f-92ac-f645b1e4e3aa None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:39.407210 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac0bb9-54b1-456f-92ac-f645b1e4e3aa None admin] RBAC: Adding query filter params (name=member) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:39.407522 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac0bb9-54b1-456f-92ac-f645b1e4e3aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:39.408911 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac0bb9-54b1-456f-92ac-f645b1e4e3aa None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:39.417641 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 113/215] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 1 16:20:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:39.423021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac0bb9-54b1-456f-92ac-f645b1e4e3aa None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:39.439507 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:39.443862 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d23c6-4cb6-481c-8c13-349df6a3c938 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:39.481063 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2e544-3998-4234-9338-80426b999271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:39.482024 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2e544-3998-4234-9338-80426b999271 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:39.482547 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2e544-3998-4234-9338-80426b999271 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:39.483061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2e544-3998-4234-9338-80426b999271 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:39.483912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2e544-3998-4234-9338-80426b999271 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:39.485620 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2e544-3998-4234-9338-80426b999271 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:39.509460 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 103/216] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 1 16:20:39 2022] POST /identity/v3/users => generated 297 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:20:39.543932 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac0bb9-54b1-456f-92ac-f645b1e4e3aa None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:39.584805 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568ed75-395e-4b9c-b2f3-52173c9b05c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:39.585868 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568ed75-395e-4b9c-b2f3-52173c9b05c2 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:39.586403 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568ed75-395e-4b9c-b2f3-52173c9b05c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:39.586877 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568ed75-395e-4b9c-b2f3-52173c9b05c2 None admin] PATH_INFO: `/v3/users/aac849f048fe412ea0e49d54985b6c6e` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:39.603827 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d568ed75-395e-4b9c-b2f3-52173c9b05c2 None admin] RBAC: Authorizing `identity:get_user(user_id=aac849f048fe412ea0e49d54985b6c6e)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:39.605657 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d568ed75-395e-4b9c-b2f3-52173c9b05c2 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:39.607876 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 114/217] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:39 2022] GET /identity/v3/users/aac849f048fe412ea0e49d54985b6c6e => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:20:39.613500 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2e544-3998-4234-9338-80426b999271 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:39.617639 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cd1c1-18ec-4a61-a7cc-b00d4935515b None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:39.617931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cd1c1-18ec-4a61-a7cc-b00d4935515b None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:39.618214 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cd1c1-18ec-4a61-a7cc-b00d4935515b None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:39.619619 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 115/218] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:20:39.625442 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758a5b7-44a9-4b80-985b-16734b900aec None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:39.625745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758a5b7-44a9-4b80-985b-16734b900aec None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:39.626004 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758a5b7-44a9-4b80-985b-16734b900aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:39.645927 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-0758a5b7-44a9-4b80-985b-16734b900aec None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:39.646992 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-0758a5b7-44a9-4b80-985b-16734b900aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:39.679383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:39.724183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 116/219] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:20:39.725292 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33f5a2-1fa9-47af-84a8-26b48251a9f6 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:39.725583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33f5a2-1fa9-47af-84a8-26b48251a9f6 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:39.726102 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33f5a2-1fa9-47af-84a8-26b48251a9f6 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:39.731456 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 117/220] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:39.737697 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011c972-edc7-4a2d-b31e-591ae2ab5acf None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:39.737697 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011c972-edc7-4a2d-b31e-591ae2ab5acf None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:39.737697 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011c972-edc7-4a2d-b31e-591ae2ab5acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:39.748060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2e544-3998-4234-9338-80426b999271 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:39.759650 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-7011c972-edc7-4a2d-b31e-591ae2ab5acf None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:39.761190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-7011c972-edc7-4a2d-b31e-591ae2ab5acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:39.797846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455dcbc5-13f5-40eb-b247-e345b66f3791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:39.798528 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455dcbc5-13f5-40eb-b247-e345b66f3791 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:39.798809 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455dcbc5-13f5-40eb-b247-e345b66f3791 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:39.799088 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455dcbc5-13f5-40eb-b247-e345b66f3791 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:39.832214 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:39.855314 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 118/221] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:20:39.889990 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455dcbc5-13f5-40eb-b247-e345b66f3791 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:39.891695 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455dcbc5-13f5-40eb-b247-e345b66f3791 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:40.004981 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 104/222] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:20:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 16:20:40.009315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7011c972-edc7-4a2d-b31e-591ae2ab5acf None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:40.010849 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455dcbc5-13f5-40eb-b247-e345b66f3791 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:40.099285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:40.164312 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455dcbc5-13f5-40eb-b247-e345b66f3791 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:40.171607 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:40.178865 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7011c972-edc7-4a2d-b31e-591ae2ab5acf None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:40.201687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f9909-34a4-41c6-8fc0-2f18b3b6f58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:40.202677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f9909-34a4-41c6-8fc0-2f18b3b6f58d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:40.203158 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f9909-34a4-41c6-8fc0-2f18b3b6f58d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:40.203626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f9909-34a4-41c6-8fc0-2f18b3b6f58d None admin] PATH_INFO: `/v3/projects/cc5b99bf5631484eb25353883854e856` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:40.205032 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995f9909-34a4-41c6-8fc0-2f18b3b6f58d None admin] RBAC: Authorizing `identity:get_project(project_id=cc5b99bf5631484eb25353883854e856)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:40.206809 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995f9909-34a4-41c6-8fc0-2f18b3b6f58d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:40.209046 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 105/223] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:40 2022] GET /identity/v3/projects/cc5b99bf5631484eb25353883854e856 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:40.210077 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f9909-34a4-41c6-8fc0-2f18b3b6f58d None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:40.220443 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00ef1a-02f3-4de4-b12f-a6db849c3fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:40.221083 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00ef1a-02f3-4de4-b12f-a6db849c3fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:40.221365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00ef1a-02f3-4de4-b12f-a6db849c3fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:40.221637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00ef1a-02f3-4de4-b12f-a6db849c3fb2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:40.232421 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00ef1a-02f3-4de4-b12f-a6db849c3fb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:40.237837 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-8b00ef1a-02f3-4de4-b12f-a6db849c3fb2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:20:40.237837 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:40.237837 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:40.237837 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:40.237837 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:40.237837 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:40.237837 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:40.237837 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:40.237837 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:40.237837 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:40.237837 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:40.237837 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:40.237837 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:40.237837 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:40.237837 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:40.237837 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:40.238731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:40.238731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:40.238731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:40.238731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:40.238731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:40.238731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:40.238731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:40.238731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:40.238731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:40.238731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:40.238731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:40.238731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:40.238731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:40.238731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:40.238731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:40.238731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:40.239484 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:40.239484 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:40.239484 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:40.239484 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:40.239484 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:40.239484 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:40.239484 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:40.239484 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:40.239484 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:40.239484 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:40.239484 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:20:40.239484 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:20:40.239484 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 119/224] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:20:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:40.240100 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00ef1a-02f3-4de4-b12f-a6db849c3fb2 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:40.330273 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f9909-34a4-41c6-8fc0-2f18b3b6f58d None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:40.351646 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00ef1a-02f3-4de4-b12f-a6db849c3fb2 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:40.357009 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:40.379149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a92714-5e81-44f2-a3c4-e4a51fe62490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:40.379851 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a92714-5e81-44f2-a3c4-e4a51fe62490 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:40.380128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a92714-5e81-44f2-a3c4-e4a51fe62490 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:40.380398 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a92714-5e81-44f2-a3c4-e4a51fe62490 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:40.393136 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a92714-5e81-44f2-a3c4-e4a51fe62490 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:40.399302 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-11a92714-5e81-44f2-a3c4-e4a51fe62490 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:20:40.399302 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:40.399302 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:40.399302 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:40.399302 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:40.399302 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:40.399302 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:40.399302 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:40.399302 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:40.399302 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:40.399302 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:40.399302 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:40.399302 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:40.399302 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:40.399302 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:40.399302 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:40.400147 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:40.400147 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:40.400147 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:40.400147 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:40.400147 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:40.400147 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:40.400147 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:40.400147 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:40.400147 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:40.400147 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:40.400147 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:40.400147 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:40.400147 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:40.400147 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:40.400147 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:40.400147 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:40.400905 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:40.400905 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:40.400905 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:40.400905 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:40.400905 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:40.400905 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:40.400905 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:40.400905 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:40.400905 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:40.400905 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:40.400905 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:20:40.400905 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:20:40.400905 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 106/225] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 1 16:20:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:40.401534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a92714-5e81-44f2-a3c4-e4a51fe62490 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:40.403594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83f6eb-1976-449b-aab4-54696ff39def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:40.404737 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83f6eb-1976-449b-aab4-54696ff39def None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:40.406133 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83f6eb-1976-449b-aab4-54696ff39def None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:40.406645 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83f6eb-1976-449b-aab4-54696ff39def None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:40.407578 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e83f6eb-1976-449b-aab4-54696ff39def None admin] RBAC: Adding query filter params (role.id=45545334e1fc473c9d85fc91a9b0a35c, scope.project.id=cc5b99bf5631484eb25353883854e856, user.id=aac849f048fe412ea0e49d54985b6c6e) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:40.408105 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e83f6eb-1976-449b-aab4-54696ff39def None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:40.409752 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e83f6eb-1976-449b-aab4-54696ff39def None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:40.417986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 120/226] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:20:40 2022] GET /identity/v3/role_assignments?user.id=aac849f048fe412ea0e49d54985b6c6e&scope.project.id=cc5b99bf5631484eb25353883854e856&role.id=45545334e1fc473c9d85fc91a9b0a35c => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:40.419064 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83f6eb-1976-449b-aab4-54696ff39def None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:40.448303 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:40.597964 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a92714-5e81-44f2-a3c4-e4a51fe62490 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:40.609101 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83f6eb-1976-449b-aab4-54696ff39def None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:40.643132 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a309fd-d2b4-449b-82a2-bb62c80ba8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:40.643751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a309fd-d2b4-449b-82a2-bb62c80ba8dd None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:40.644029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a309fd-d2b4-449b-82a2-bb62c80ba8dd None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:40.644296 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a309fd-d2b4-449b-82a2-bb62c80ba8dd None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:40.644956 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a309fd-d2b4-449b-82a2-bb62c80ba8dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:40.645311 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a309fd-d2b4-449b-82a2-bb62c80ba8dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:40.646964 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a309fd-d2b4-449b-82a2-bb62c80ba8dd None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:40.652938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d730815a-e8c6-42e9-9225-219abb9526e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:40.653818 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730815a-e8c6-42e9-9225-219abb9526e7 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:40.654336 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730815a-e8c6-42e9-9225-219abb9526e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:40.655324 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730815a-e8c6-42e9-9225-219abb9526e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:40.656181 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d730815a-e8c6-42e9-9225-219abb9526e7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:40.656708 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d730815a-e8c6-42e9-9225-219abb9526e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:40.658396 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d730815a-e8c6-42e9-9225-219abb9526e7 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:40.665784 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 107/227] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:40.667763 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 121/228] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:20:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:40.673612 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a309fd-d2b4-449b-82a2-bb62c80ba8dd None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:40.674693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d730815a-e8c6-42e9-9225-219abb9526e7 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:40.722161 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:40.769719 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:40.792810 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d730815a-e8c6-42e9-9225-219abb9526e7 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:40.829633 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae0e4e8-f571-4ed3-b474-3b7158c3a7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:40.830656 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0e4e8-f571-4ed3-b474-3b7158c3a7e5 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:40.831133 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0e4e8-f571-4ed3-b474-3b7158c3a7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:40.831715 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0e4e8-f571-4ed3-b474-3b7158c3a7e5 None admin] PATH_INFO: `/v3/users/08bdf9a0060d472fa267472d8bbe5585` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:40.833155 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae0e4e8-f571-4ed3-b474-3b7158c3a7e5 None admin] RBAC: Authorizing `identity:get_user(user_id=08bdf9a0060d472fa267472d8bbe5585)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:40.834862 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae0e4e8-f571-4ed3-b474-3b7158c3a7e5 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:40.837001 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 122/229] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:40 2022] GET /identity/v3/users/08bdf9a0060d472fa267472d8bbe5585 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:40.844842 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a309fd-d2b4-449b-82a2-bb62c80ba8dd None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:40.846718 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae0e4e8-f571-4ed3-b474-3b7158c3a7e5 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:40.893969 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba555ff-2fb7-4a4c-979f-ebd30dbcaa2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:40.894707 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba555ff-2fb7-4a4c-979f-ebd30dbcaa2a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:40.894993 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba555ff-2fb7-4a4c-979f-ebd30dbcaa2a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:40.895399 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba555ff-2fb7-4a4c-979f-ebd30dbcaa2a None admin] PATH_INFO: `/v3/users/08bdf9a0060d472fa267472d8bbe5585` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:40.901230 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba555ff-2fb7-4a4c-979f-ebd30dbcaa2a None admin] RBAC: Authorizing `identity:get_user(user_id=08bdf9a0060d472fa267472d8bbe5585)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:40.902057 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba555ff-2fb7-4a4c-979f-ebd30dbcaa2a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:40.907173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 108/230] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:40 2022] GET /identity/v3/users/08bdf9a0060d472fa267472d8bbe5585 => generated 283 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:40.909621 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba555ff-2fb7-4a4c-979f-ebd30dbcaa2a None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:41.066038 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba555ff-2fb7-4a4c-979f-ebd30dbcaa2a None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:41.070492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae0e4e8-f571-4ed3-b474-3b7158c3a7e5 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:41.101977 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:41.111416 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eca77f-af6f-436c-ab4e-5233e06e742c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:41.112054 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eca77f-af6f-436c-ab4e-5233e06e742c None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:41.112331 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eca77f-af6f-436c-ab4e-5233e06e742c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:41.112606 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eca77f-af6f-436c-ab4e-5233e06e742c None admin] PATH_INFO: `/v3/projects/cc5b99bf5631484eb25353883854e856` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:41.113740 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eca77f-af6f-436c-ab4e-5233e06e742c None admin] RBAC: Authorizing `identity:get_project(project_id=cc5b99bf5631484eb25353883854e856)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:41.115203 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eca77f-af6f-436c-ab4e-5233e06e742c None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:41.119382 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 109/231] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:40 2022] GET /identity/v3/projects/cc5b99bf5631484eb25353883854e856 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:41.125902 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eca77f-af6f-436c-ab4e-5233e06e742c None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:41.131570 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df450d-0839-42bf-880b-55ebc3eb3713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:41.132605 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df450d-0839-42bf-880b-55ebc3eb3713 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:41.132605 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df450d-0839-42bf-880b-55ebc3eb3713 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:41.132787 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df450d-0839-42bf-880b-55ebc3eb3713 None admin] PATH_INFO: `/v3/projects/cc5b99bf5631484eb25353883854e856` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:41.133943 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df450d-0839-42bf-880b-55ebc3eb3713 None admin] RBAC: Authorizing `identity:get_project(project_id=cc5b99bf5631484eb25353883854e856)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:41.135413 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df450d-0839-42bf-880b-55ebc3eb3713 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:41.141156 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 123/232] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:40 2022] GET /identity/v3/projects/cc5b99bf5631484eb25353883854e856 => generated 302 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:41.143231 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df450d-0839-42bf-880b-55ebc3eb3713 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:41.161788 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:41.267188 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df450d-0839-42bf-880b-55ebc3eb3713 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:41.302765 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3acca-6574-4db8-9b57-d7a2ac3b71f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:41.303695 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3acca-6574-4db8-9b57-d7a2ac3b71f4 None admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:41.303821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3acca-6574-4db8-9b57-d7a2ac3b71f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:41.303998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3acca-6574-4db8-9b57-d7a2ac3b71f4 None admin] PATH_INFO: `/v3/projects/cc5b99bf5631484eb25353883854e856/users/08bdf9a0060d472fa267472d8bbe5585/roles/f2a6e936aa294323ba7087bf377276b2` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:41.306195 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3acca-6574-4db8-9b57-d7a2ac3b71f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc5b99bf5631484eb25353883854e856, user_id=08bdf9a0060d472fa267472d8bbe5585, role_id=f2a6e936aa294323ba7087bf377276b2)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:41.307675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3acca-6574-4db8-9b57-d7a2ac3b71f4 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:41.313388 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eca77f-af6f-436c-ab4e-5233e06e742c None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:41.331789 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 124/233] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 1 16:20:41 2022] PUT /identity/v3/projects/cc5b99bf5631484eb25353883854e856/users/08bdf9a0060d472fa267472d8bbe5585/roles/f2a6e936aa294323ba7087bf377276b2 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:20:41.374986 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4673a-41a3-4446-a02e-a8ecb0b6f5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:41.375947 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4673a-41a3-4446-a02e-a8ecb0b6f5ff None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:41.376436 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4673a-41a3-4446-a02e-a8ecb0b6f5ff None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:41.376906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4673a-41a3-4446-a02e-a8ecb0b6f5ff None admin] PATH_INFO: `/v3/projects/cc5b99bf5631484eb25353883854e856/users/08bdf9a0060d472fa267472d8bbe5585/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:41.379304 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae4673a-41a3-4446-a02e-a8ecb0b6f5ff None admin] RBAC: Authorizing `identity:create_grant(project_id=cc5b99bf5631484eb25353883854e856, user_id=08bdf9a0060d472fa267472d8bbe5585, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:41.381062 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae4673a-41a3-4446-a02e-a8ecb0b6f5ff None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:41.410718 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 110/234] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 1 16:20:41 2022] PUT /identity/v3/projects/cc5b99bf5631484eb25353883854e856/users/08bdf9a0060d472fa267472d8bbe5585/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:20:41.973540 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda6982-63bf-4f41-99f5-b98ff1c05773 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:41.974071 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda6982-63bf-4f41-99f5-b98ff1c05773 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:41.974152 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda6982-63bf-4f41-99f5-b98ff1c05773 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:41.975626 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 125/235] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:41.980901 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f5257-a1cf-41c8-9211-b1b3e436dd0a None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:41.981333 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f5257-a1cf-41c8-9211-b1b3e436dd0a None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:41.981512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f5257-a1cf-41c8-9211-b1b3e436dd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:42.000597 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-845f5257-a1cf-41c8-9211-b1b3e436dd0a None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:42.001786 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-845f5257-a1cf-41c8-9211-b1b3e436dd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:42.050625 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:42.092571 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 111/236] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:42.114815 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72899ff9-1e8e-4193-94cb-5e01066746fc None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:42.115107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72899ff9-1e8e-4193-94cb-5e01066746fc None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:42.115386 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72899ff9-1e8e-4193-94cb-5e01066746fc None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:42.116948 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 126/237] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:42.122210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d557e3-7aed-4354-808f-a89fda0199a5 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:42.122525 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d557e3-7aed-4354-808f-a89fda0199a5 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:42.122797 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d557e3-7aed-4354-808f-a89fda0199a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:42.148309 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-54d557e3-7aed-4354-808f-a89fda0199a5 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:42.149530 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-54d557e3-7aed-4354-808f-a89fda0199a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:42.236909 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 112/238] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:42.428801 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72899ff9-1e8e-4193-94cb-5e01066746fc None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:42.528199 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:42.559893 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72899ff9-1e8e-4193-94cb-5e01066746fc None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:42.602475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d557e3-7aed-4354-808f-a89fda0199a5 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:42.604393 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d3a23-af65-4e6d-9894-553e984e2a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:42.605360 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d3a23-af65-4e6d-9894-553e984e2a94 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:42.606579 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d3a23-af65-4e6d-9894-553e984e2a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:42.606874 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d3a23-af65-4e6d-9894-553e984e2a94 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:42.619008 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d3a23-af65-4e6d-9894-553e984e2a94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:42.625823 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-5c3d3a23-af65-4e6d-9894-553e984e2a94 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:20:42.625823 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:42.625823 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:42.625823 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:42.625823 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:42.625823 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:42.625823 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:42.625823 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:42.625823 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:42.625823 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:42.625823 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:42.625823 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:42.625823 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:42.625823 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:42.625823 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:42.625823 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:42.626711 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:42.626711 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:42.626711 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:42.626711 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:42.626711 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:42.626711 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:42.626711 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:42.626711 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:42.626711 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:42.626711 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:42.626711 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:42.626711 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:42.626711 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:42.626711 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:42.626711 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:42.626711 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:42.627470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:42.627470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:42.627470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:42.627470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:42.627470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:42.627470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:42.627470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:42.627470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:42.627470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:42.627470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:42.627470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:20:42.627470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:20:42.627470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 127/239] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:42.632601 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d3a23-af65-4e6d-9894-553e984e2a94 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:42.755545 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:42.760427 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d557e3-7aed-4354-808f-a89fda0199a5 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:42.793878 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d3a23-af65-4e6d-9894-553e984e2a94 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:42.807818 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef29b3b-b324-4819-8de5-fefdfc7ddd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:42.808992 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef29b3b-b324-4819-8de5-fefdfc7ddd04 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:42.809940 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef29b3b-b324-4819-8de5-fefdfc7ddd04 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:42.810542 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef29b3b-b324-4819-8de5-fefdfc7ddd04 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:42.812193 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef29b3b-b324-4819-8de5-fefdfc7ddd04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:42.814203 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef29b3b-b324-4819-8de5-fefdfc7ddd04 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:42.816760 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 113/240] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:20:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:42.818118 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87247e-6da1-4aed-83aa-0e70e2efbd64 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:42.818599 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87247e-6da1-4aed-83aa-0e70e2efbd64 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:42.819028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87247e-6da1-4aed-83aa-0e70e2efbd64 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:42.820389 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 114/241] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:42.824528 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee87247e-6da1-4aed-83aa-0e70e2efbd64 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:42.839903 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b23806-1356-490e-b6a7-a0a3458230f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:42.840522 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b23806-1356-490e-b6a7-a0a3458230f3 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:42.840861 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b23806-1356-490e-b6a7-a0a3458230f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:42.841161 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b23806-1356-490e-b6a7-a0a3458230f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:42.842552 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b23806-1356-490e-b6a7-a0a3458230f3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:42.842866 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b23806-1356-490e-b6a7-a0a3458230f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:42.844943 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b23806-1356-490e-b6a7-a0a3458230f3 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:42.848264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:42.859375 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 128/242] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 1 16:20:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:20:42.861289 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b77be-1321-4c91-8042-b3d4a4cdab14 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:42.861411 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b77be-1321-4c91-8042-b3d4a4cdab14 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:42.861654 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b77be-1321-4c91-8042-b3d4a4cdab14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:42.881225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-7d3b77be-1321-4c91-8042-b3d4a4cdab14 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:42.882556 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-7d3b77be-1321-4c91-8042-b3d4a4cdab14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:42.958611 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 129/243] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:42.959585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b77be-1321-4c91-8042-b3d4a4cdab14 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:42.959922 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee87247e-6da1-4aed-83aa-0e70e2efbd64 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:43.004325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa16a4c-5cd0-4e1d-b054-3a0e2ec1449a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:43.006135 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa16a4c-5cd0-4e1d-b054-3a0e2ec1449a None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:43.007030 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa16a4c-5cd0-4e1d-b054-3a0e2ec1449a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:43.007506 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa16a4c-5cd0-4e1d-b054-3a0e2ec1449a None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:43.008593 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa16a4c-5cd0-4e1d-b054-3a0e2ec1449a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:43.010916 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa16a4c-5cd0-4e1d-b054-3a0e2ec1449a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:43.038579 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 115/244] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 1 16:20:42 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:20:43.112588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b77be-1321-4c91-8042-b3d4a4cdab14 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:43.149971 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:43.162195 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a0450-a196-4565-9da3-d983c631f576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:43.162944 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a0450-a196-4565-9da3-d983c631f576 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:43.163228 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a0450-a196-4565-9da3-d983c631f576 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:43.163553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a0450-a196-4565-9da3-d983c631f576 None admin] PATH_INFO: `/v3/users/e3edf84518fd4cfd849f110ba2ff7217` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:43.178698 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a0450-a196-4565-9da3-d983c631f576 None admin] RBAC: Authorizing `identity:get_user(user_id=e3edf84518fd4cfd849f110ba2ff7217)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:43.180258 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a0450-a196-4565-9da3-d983c631f576 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:43.182439 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 130/245] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:42 2022] GET /identity/v3/users/e3edf84518fd4cfd849f110ba2ff7217 => generated 297 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:43.187829 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa16a4c-5cd0-4e1d-b054-3a0e2ec1449a None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:43.233056 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:43.317853 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa16a4c-5cd0-4e1d-b054-3a0e2ec1449a None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:43.358257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f065b6-e8f8-4328-8356-6965991083c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:43.358901 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f065b6-e8f8-4328-8356-6965991083c5 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:43.359183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f065b6-e8f8-4328-8356-6965991083c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:43.359457 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f065b6-e8f8-4328-8356-6965991083c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:43.386797 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a0450-a196-4565-9da3-d983c631f576 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:43.447957 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f065b6-e8f8-4328-8356-6965991083c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:43.449812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f065b6-e8f8-4328-8356-6965991083c5 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:43.483847 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:43.516285 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a0450-a196-4565-9da3-d983c631f576 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:43.558056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c95f5-aff3-4865-a5fb-069777a68d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:43.559228 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c95f5-aff3-4865-a5fb-069777a68d7a None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:43.559745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c95f5-aff3-4865-a5fb-069777a68d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:43.560229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c95f5-aff3-4865-a5fb-069777a68d7a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:43.573057 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29c95f5-aff3-4865-a5fb-069777a68d7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:43.579580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-f29c95f5-aff3-4865-a5fb-069777a68d7a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:20:43.579580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:43.579580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:43.579580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:43.579580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:43.579580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:43.579580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:43.579580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:43.579580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:43.579580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:43.579580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:43.579580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:43.579580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:43.579580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:43.579580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:43.579580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:43.580454 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:43.580454 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:43.580454 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:43.580454 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:43.580454 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:43.580454 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:43.580454 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:43.580454 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:43.580454 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:43.580454 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:43.580454 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:43.580454 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:43.580454 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:43.580454 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:43.580454 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:43.580454 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:43.587514 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:43.587514 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:43.587514 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:43.587514 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:43.587514 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:43.587514 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:43.587514 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:43.587514 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:43.587514 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:43.587514 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:43.587514 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:20:43.587514 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:20:43.588130 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 116/246] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:20:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:20:43.588207 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 131/247] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:43.588271 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f065b6-e8f8-4328-8356-6965991083c5 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:43.591186 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c95f5-aff3-4865-a5fb-069777a68d7a None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:43.654050 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:43.719081 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f065b6-e8f8-4328-8356-6965991083c5 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:43.726293 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c95f5-aff3-4865-a5fb-069777a68d7a None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:43.749768 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:43.765169 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71f29d-00fa-49da-9fbb-84ecb19a95da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:43.765917 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71f29d-00fa-49da-9fbb-84ecb19a95da None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:43.766198 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71f29d-00fa-49da-9fbb-84ecb19a95da None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:43.766501 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71f29d-00fa-49da-9fbb-84ecb19a95da None admin] PATH_INFO: `/v3/projects/cc5b99bf5631484eb25353883854e856` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:43.767646 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da71f29d-00fa-49da-9fbb-84ecb19a95da None admin] RBAC: Authorizing `identity:get_project(project_id=cc5b99bf5631484eb25353883854e856)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:43.769082 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da71f29d-00fa-49da-9fbb-84ecb19a95da None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:43.770769 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 117/248] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:43 2022] GET /identity/v3/projects/cc5b99bf5631484eb25353883854e856 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:43.771894 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e1d3d-6ba7-4a1f-a5a6-0f54760bf649 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:43.772175 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e1d3d-6ba7-4a1f-a5a6-0f54760bf649 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:43.772448 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e1d3d-6ba7-4a1f-a5a6-0f54760bf649 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:43.773638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 118/249] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:43.774683 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485df00-9609-4afd-bfa2-2c64d84b74ef None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:43.774959 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485df00-9609-4afd-bfa2-2c64d84b74ef None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:43.775227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485df00-9609-4afd-bfa2-2c64d84b74ef None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:43.776465 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 119/250] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:43.778356 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07def9dd-29fd-43c0-9cb4-f6654b9cc998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:43.778682 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e485df00-9609-4afd-bfa2-2c64d84b74ef None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:43.778988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07def9dd-29fd-43c0-9cb4-f6654b9cc998 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:43.779274 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07def9dd-29fd-43c0-9cb4-f6654b9cc998 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:43.779594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07def9dd-29fd-43c0-9cb4-f6654b9cc998 None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:43.780278 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07def9dd-29fd-43c0-9cb4-f6654b9cc998 None admin] RBAC: Adding query filter params (name=member) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:43.780587 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07def9dd-29fd-43c0-9cb4-f6654b9cc998 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:43.781973 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07def9dd-29fd-43c0-9cb4-f6654b9cc998 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:43.792063 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 132/251] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 1 16:20:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:43.793297 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445439f5-c854-43b7-8b91-376bb754dec5 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:43.793589 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445439f5-c854-43b7-8b91-376bb754dec5 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:43.793863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445439f5-c854-43b7-8b91-376bb754dec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:43.814873 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-445439f5-c854-43b7-8b91-376bb754dec5 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:43.816061 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-445439f5-c854-43b7-8b91-376bb754dec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:43.900812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e485df00-9609-4afd-bfa2-2c64d84b74ef None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:43.911596 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 133/252] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:43.913652 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94747e6-37ac-4796-8026-fc7a4e026782 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:43.914172 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94747e6-37ac-4796-8026-fc7a4e026782 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:43.914962 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94747e6-37ac-4796-8026-fc7a4e026782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:43.938910 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-d94747e6-37ac-4796-8026-fc7a4e026782 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:43.940798 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-d94747e6-37ac-4796-8026-fc7a4e026782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:43.963362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e312c06-5771-46d1-ba12-edc1da7e31e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:43.963981 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e312c06-5771-46d1-ba12-edc1da7e31e3 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:43.964261 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e312c06-5771-46d1-ba12-edc1da7e31e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:43.964535 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e312c06-5771-46d1-ba12-edc1da7e31e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:43.965282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e312c06-5771-46d1-ba12-edc1da7e31e3 None admin] RBAC: Adding query filter params (role.id=12fe0aa7f1174edfb109315b88d6267b, scope.project.id=cc5b99bf5631484eb25353883854e856, user.id=e3edf84518fd4cfd849f110ba2ff7217) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:43.965593 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e312c06-5771-46d1-ba12-edc1da7e31e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:43.967098 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e312c06-5771-46d1-ba12-edc1da7e31e3 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:43.981129 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 120/253] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:20:43 2022] GET /identity/v3/role_assignments?user.id=e3edf84518fd4cfd849f110ba2ff7217&scope.project.id=cc5b99bf5631484eb25353883854e856&role.id=12fe0aa7f1174edfb109315b88d6267b => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:43.983002 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e312c06-5771-46d1-ba12-edc1da7e31e3 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:43.990346 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:44.036488 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 134/254] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:44.113382 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:44.172187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e312c06-5771-46d1-ba12-edc1da7e31e3 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:44.219724 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439c7e3-0a83-4302-844b-256423a96ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:44.220387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439c7e3-0a83-4302-844b-256423a96ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:44.220670 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439c7e3-0a83-4302-844b-256423a96ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:44.220947 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439c7e3-0a83-4302-844b-256423a96ec9 None admin] PATH_INFO: `/v3/users/aac849f048fe412ea0e49d54985b6c6e` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:44.222131 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439c7e3-0a83-4302-844b-256423a96ec9 None admin] RBAC: Authorizing `identity:get_user(user_id=aac849f048fe412ea0e49d54985b6c6e)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:44.223649 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439c7e3-0a83-4302-844b-256423a96ec9 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:44.225304 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 121/255] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:43 2022] GET /identity/v3/users/aac849f048fe412ea0e49d54985b6c6e => generated 297 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:44.231155 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94747e6-37ac-4796-8026-fc7a4e026782 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:44.275387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439c7e3-0a83-4302-844b-256423a96ec9 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:44.373907 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94747e6-37ac-4796-8026-fc7a4e026782 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:44.428643 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d578e3-b765-4c66-a5d3-249aeda067be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:44.429504 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d578e3-b765-4c66-a5d3-249aeda067be None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:44.434614 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d578e3-b765-4c66-a5d3-249aeda067be None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:44.435072 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d578e3-b765-4c66-a5d3-249aeda067be None admin] PATH_INFO: `/v3/projects/cc5b99bf5631484eb25353883854e856` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:44.436939 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d578e3-b765-4c66-a5d3-249aeda067be None admin] RBAC: Authorizing `identity:get_project(project_id=cc5b99bf5631484eb25353883854e856)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:44.438573 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d578e3-b765-4c66-a5d3-249aeda067be None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:44.440366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 135/256] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:44 2022] GET /identity/v3/projects/cc5b99bf5631484eb25353883854e856 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:44.446113 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d578e3-b765-4c66-a5d3-249aeda067be None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:44.458262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439c7e3-0a83-4302-844b-256423a96ec9 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:44.494711 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:44.518642 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:44.531129 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc28e691-56f6-40b3-bb4f-fc1b51beabd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:44.531798 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc28e691-56f6-40b3-bb4f-fc1b51beabd9 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:44.532084 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc28e691-56f6-40b3-bb4f-fc1b51beabd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:44.532360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc28e691-56f6-40b3-bb4f-fc1b51beabd9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:44.547164 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc28e691-56f6-40b3-bb4f-fc1b51beabd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:44.564639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-fc28e691-56f6-40b3-bb4f-fc1b51beabd9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:20:44.564639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:44.564639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:44.564639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:44.564639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:44.564639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:44.564639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:44.564639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:44.564639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:44.564639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:44.564639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:44.564639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:44.564639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:44.564639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:44.564639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:44.564639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:44.565519 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:44.565519 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:44.565519 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:44.565519 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:44.565519 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:44.565519 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:44.565519 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:44.565519 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:44.565519 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:44.565519 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:44.565519 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:44.565519 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:44.565519 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:44.565519 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:44.565519 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:44.565519 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:44.566325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:44.566325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:44.566325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:44.566325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:44.566325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:44.566325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:44.566325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:44.566325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:44.566325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:44.566325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:44.566325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:20:44.566325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:20:44.566325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 122/257] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 1 16:20:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:20:44.579189 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc28e691-56f6-40b3-bb4f-fc1b51beabd9 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:44.606312 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d578e3-b765-4c66-a5d3-249aeda067be None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:44.674947 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8a09c-3688-4fd9-b423-8e4a7584201e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:44.675619 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8a09c-3688-4fd9-b423-8e4a7584201e None admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:44.675900 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8a09c-3688-4fd9-b423-8e4a7584201e None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:44.676175 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8a09c-3688-4fd9-b423-8e4a7584201e None admin] PATH_INFO: `/v3/projects/cc5b99bf5631484eb25353883854e856/users/aac849f048fe412ea0e49d54985b6c6e/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:44.680150 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de8a09c-3688-4fd9-b423-8e4a7584201e None admin] RBAC: Authorizing `identity:create_grant(project_id=cc5b99bf5631484eb25353883854e856, user_id=aac849f048fe412ea0e49d54985b6c6e, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:44.681555 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de8a09c-3688-4fd9-b423-8e4a7584201e None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:44.704796 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 136/258] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 1 16:20:44 2022] PUT /identity/v3/projects/cc5b99bf5631484eb25353883854e856/users/aac849f048fe412ea0e49d54985b6c6e/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:20:44.705622 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8a09c-3688-4fd9-b423-8e4a7584201e None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:44.811305 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc28e691-56f6-40b3-bb4f-fc1b51beabd9 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:44.891952 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5196898e-5e86-4e10-86ea-330ce8df8d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:44.892610 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196898e-5e86-4e10-86ea-330ce8df8d5f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:44.892894 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196898e-5e86-4e10-86ea-330ce8df8d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:44.893157 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196898e-5e86-4e10-86ea-330ce8df8d5f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:44.923107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5196898e-5e86-4e10-86ea-330ce8df8d5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:44.929017 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-5196898e-5e86-4e10-86ea-330ce8df8d5f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:20:44.929017 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:44.929017 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:44.929017 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:44.929017 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:44.929017 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:44.929017 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:44.929017 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:44.929017 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:44.929017 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:44.929017 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:44.929017 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:44.929017 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:44.929017 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:44.929017 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:44.929017 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:44.931120 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:44.931120 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:44.931120 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:44.931120 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:44.931120 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:44.931120 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:44.931120 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:44.931120 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:44.931120 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:44.931120 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:44.931120 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:44.931120 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:44.931120 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:44.931120 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:44.931120 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:44.931120 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:44.933049 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:44.933049 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:44.933049 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:44.933049 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:44.933049 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:44.933049 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:44.933049 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:44.933049 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:44.933049 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:44.933049 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:44.933049 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:20:44.933049 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:20:44.933049 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 123/259] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:20:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:44.936763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5196898e-5e86-4e10-86ea-330ce8df8d5f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:44.969669 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:44.988278 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8a09c-3688-4fd9-b423-8e4a7584201e None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:45.041602 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:45.080771 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb2380-f8aa-45aa-8653-4bf51e209d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:45.081404 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb2380-f8aa-45aa-8653-4bf51e209d2f None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:45.081687 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb2380-f8aa-45aa-8653-4bf51e209d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:45.081965 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb2380-f8aa-45aa-8653-4bf51e209d2f None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:45.082674 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cb2380-f8aa-45aa-8653-4bf51e209d2f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:45.082983 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cb2380-f8aa-45aa-8653-4bf51e209d2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:45.084369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cb2380-f8aa-45aa-8653-4bf51e209d2f None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:45.097747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 137/260] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:20:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:45.104575 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb2380-f8aa-45aa-8653-4bf51e209d2f None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:45.157167 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5196898e-5e86-4e10-86ea-330ce8df8d5f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:45.243660 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5f683-15a4-4ea5-b940-7c200cab38bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:45.244621 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5f683-15a4-4ea5-b940-7c200cab38bb None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:45.245102 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5f683-15a4-4ea5-b940-7c200cab38bb None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:45.245608 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5f683-15a4-4ea5-b940-7c200cab38bb None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:45.246570 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f5f683-15a4-4ea5-b940-7c200cab38bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:45.247078 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f5f683-15a4-4ea5-b940-7c200cab38bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:45.248973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f5f683-15a4-4ea5-b940-7c200cab38bb None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:45.254846 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb2380-f8aa-45aa-8653-4bf51e209d2f None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:45.272172 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 124/261] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:20:45.281792 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5f683-15a4-4ea5-b940-7c200cab38bb None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:45.291273 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceffa87-e0d1-4a25-b936-7b5649882d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:45.291920 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceffa87-e0d1-4a25-b936-7b5649882d95 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:45.292203 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceffa87-e0d1-4a25-b936-7b5649882d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:45.292474 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceffa87-e0d1-4a25-b936-7b5649882d95 None admin] PATH_INFO: `/v3/users/08bdf9a0060d472fa267472d8bbe5585` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:45.293625 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceffa87-e0d1-4a25-b936-7b5649882d95 None admin] RBAC: Authorizing `identity:get_user(user_id=08bdf9a0060d472fa267472d8bbe5585)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:45.295653 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceffa87-e0d1-4a25-b936-7b5649882d95 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:45.297519 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 138/262] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:45 2022] GET /identity/v3/users/08bdf9a0060d472fa267472d8bbe5585 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:45.308427 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceffa87-e0d1-4a25-b936-7b5649882d95 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:45.362596 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:45.463820 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5f683-15a4-4ea5-b940-7c200cab38bb None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:45.472461 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:45.506343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080c337-1705-44d3-a631-84157e42a344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:45.507656 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080c337-1705-44d3-a631-84157e42a344 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:45.508256 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080c337-1705-44d3-a631-84157e42a344 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:45.508858 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080c337-1705-44d3-a631-84157e42a344 None admin] PATH_INFO: `/v3/users/08bdf9a0060d472fa267472d8bbe5585` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:45.510367 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2080c337-1705-44d3-a631-84157e42a344 None admin] RBAC: Authorizing `identity:get_user(user_id=08bdf9a0060d472fa267472d8bbe5585)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:45.512491 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2080c337-1705-44d3-a631-84157e42a344 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:45.515271 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceffa87-e0d1-4a25-b936-7b5649882d95 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:45.515500 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 125/263] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:45 2022] GET /identity/v3/users/08bdf9a0060d472fa267472d8bbe5585 => generated 283 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:45.524104 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080c337-1705-44d3-a631-84157e42a344 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:45.602960 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e20e41-406e-45d5-8372-9992a1f3bc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:45.603743 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e20e41-406e-45d5-8372-9992a1f3bc0b None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:45.604132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e20e41-406e-45d5-8372-9992a1f3bc0b None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:45.604551 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e20e41-406e-45d5-8372-9992a1f3bc0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:45.655404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080c337-1705-44d3-a631-84157e42a344 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:45.696738 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5efc90-11b8-4b5c-aa0f-0109f22536e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:45.697598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5efc90-11b8-4b5c-aa0f-0109f22536e9 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:45.698449 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5efc90-11b8-4b5c-aa0f-0109f22536e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:45.698872 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5efc90-11b8-4b5c-aa0f-0109f22536e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:45.742007 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e20e41-406e-45d5-8372-9992a1f3bc0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:45.743711 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e20e41-406e-45d5-8372-9992a1f3bc0b None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:45.752176 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:45.784163 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5efc90-11b8-4b5c-aa0f-0109f22536e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:45.786631 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5efc90-11b8-4b5c-aa0f-0109f22536e9 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:45.797207 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:45.909099 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 126/264] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:20:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:20:45.914888 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5efc90-11b8-4b5c-aa0f-0109f22536e9 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:45.973136 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 139/265] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:20:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 666 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:20:45.996080 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e20e41-406e-45d5-8372-9992a1f3bc0b None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:46.006781 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:46.035672 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5efc90-11b8-4b5c-aa0f-0109f22536e9 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:46.072594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8d4b0-0b12-4175-85e9-f6df8118d2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:46.073662 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8d4b0-0b12-4175-85e9-f6df8118d2c7 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:46.074126 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8d4b0-0b12-4175-85e9-f6df8118d2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:46.074634 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8d4b0-0b12-4175-85e9-f6df8118d2c7 None admin] PATH_INFO: `/v3/projects/cc5b99bf5631484eb25353883854e856` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:46.075956 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8d4b0-0b12-4175-85e9-f6df8118d2c7 None admin] RBAC: Authorizing `identity:get_project(project_id=cc5b99bf5631484eb25353883854e856)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:46.078373 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8d4b0-0b12-4175-85e9-f6df8118d2c7 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:46.080568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 127/266] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:45 2022] GET /identity/v3/projects/cc5b99bf5631484eb25353883854e856 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:46.085804 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8d4b0-0b12-4175-85e9-f6df8118d2c7 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:46.187454 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e20e41-406e-45d5-8372-9992a1f3bc0b None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:46.239865 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:46.257998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec72e646-1c46-4b21-a194-ecb67c19eabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:46.258764 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72e646-1c46-4b21-a194-ecb67c19eabf None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:46.258951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72e646-1c46-4b21-a194-ecb67c19eabf None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:46.259229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72e646-1c46-4b21-a194-ecb67c19eabf None admin] PATH_INFO: `/v3/projects/cc5b99bf5631484eb25353883854e856` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:46.261384 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec72e646-1c46-4b21-a194-ecb67c19eabf None admin] RBAC: Authorizing `identity:get_project(project_id=cc5b99bf5631484eb25353883854e856)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:46.262877 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec72e646-1c46-4b21-a194-ecb67c19eabf None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:46.264557 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 140/267] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:45 2022] GET /identity/v3/projects/cc5b99bf5631484eb25353883854e856 => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:46.265687 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8af99f-be00-45b1-ba4f-884b100e6cbe None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:46.265974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8af99f-be00-45b1-ba4f-884b100e6cbe None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:46.266268 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8af99f-be00-45b1-ba4f-884b100e6cbe None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:46.270024 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8d4b0-0b12-4175-85e9-f6df8118d2c7 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:46.273893 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 141/268] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:46.274087 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8af99f-be00-45b1-ba4f-884b100e6cbe None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:46.335666 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad80a4a-c52b-429b-9955-6d965a64acf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:46.336219 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad80a4a-c52b-429b-9955-6d965a64acf5 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:46.336500 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad80a4a-c52b-429b-9955-6d965a64acf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:46.336774 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad80a4a-c52b-429b-9955-6d965a64acf5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:46.337520 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad80a4a-c52b-429b-9955-6d965a64acf5 None admin] RBAC: Adding query filter params (role.id=e8ba063905484d22993bcf04830091a3, scope.project.id=cc5b99bf5631484eb25353883854e856, user.id=08bdf9a0060d472fa267472d8bbe5585) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:46.337829 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad80a4a-c52b-429b-9955-6d965a64acf5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:46.339347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad80a4a-c52b-429b-9955-6d965a64acf5 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:46.364365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 128/269] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:20:46 2022] GET /identity/v3/role_assignments?user.id=08bdf9a0060d472fa267472d8bbe5585&scope.project.id=cc5b99bf5631484eb25353883854e856&role.id=e8ba063905484d22993bcf04830091a3 => generated 608 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:20:46.368932 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d5090-1eb9-4193-8ddc-04f73e0270f7 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:46.374267 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d5090-1eb9-4193-8ddc-04f73e0270f7 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:46.374267 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d5090-1eb9-4193-8ddc-04f73e0270f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:46.399649 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:46.410316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-0a1d5090-1eb9-4193-8ddc-04f73e0270f7 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:46.411541 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0a1d5090-1eb9-4193-8ddc-04f73e0270f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:46.483833 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8af99f-be00-45b1-ba4f-884b100e6cbe None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:46.525356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 129/270] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:46.553050 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc07d34-7e0a-49fb-8587-ce933be2d0f0 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:46.553502 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc07d34-7e0a-49fb-8587-ce933be2d0f0 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:46.553908 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc07d34-7e0a-49fb-8587-ce933be2d0f0 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:46.555932 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 130/271] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:20:46.561841 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabb374-c648-46b4-ac27-2abfff9eaa88 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:46.562135 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabb374-c648-46b4-ac27-2abfff9eaa88 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:46.562792 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabb374-c648-46b4-ac27-2abfff9eaa88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:46.567430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5877ac-5f51-446a-a71b-852fa0b6d1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:46.568051 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5877ac-5f51-446a-a71b-852fa0b6d1dd None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:46.568335 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5877ac-5f51-446a-a71b-852fa0b6d1dd None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:46.568614 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5877ac-5f51-446a-a71b-852fa0b6d1dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:46.569342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5877ac-5f51-446a-a71b-852fa0b6d1dd None admin] RBAC: Adding query filter params (role.id=f2a6e936aa294323ba7087bf377276b2, scope.project.id=cc5b99bf5631484eb25353883854e856, user.id=08bdf9a0060d472fa267472d8bbe5585) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:46.569658 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5877ac-5f51-446a-a71b-852fa0b6d1dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:46.571373 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5877ac-5f51-446a-a71b-852fa0b6d1dd None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:46.579125 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 142/272] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:20:46 2022] GET /identity/v3/role_assignments?user.id=08bdf9a0060d472fa267472d8bbe5585&scope.project.id=cc5b99bf5631484eb25353883854e856&role.id=f2a6e936aa294323ba7087bf377276b2 => generated 608 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:46.597223 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-bfabb374-c648-46b4-ac27-2abfff9eaa88 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:46.598434 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-bfabb374-c648-46b4-ac27-2abfff9eaa88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:46.708051 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 131/273] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:46.789572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1311a-359c-47ce-9598-574719842d4c None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:46.789901 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1311a-359c-47ce-9598-574719842d4c None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:46.790190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1311a-359c-47ce-9598-574719842d4c None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:46.791847 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 143/274] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:46.797609 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569bc63-453f-4004-a2a0-c9eba075cde2 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:46.797901 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569bc63-453f-4004-a2a0-c9eba075cde2 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:46.798182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569bc63-453f-4004-a2a0-c9eba075cde2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:46.822950 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-2569bc63-453f-4004-a2a0-c9eba075cde2 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:46.824171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2569bc63-453f-4004-a2a0-c9eba075cde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:46.873778 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c1311a-359c-47ce-9598-574719842d4c None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:46.912119 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:46.924902 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 132/275] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:46.965375 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:47.014217 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c1311a-359c-47ce-9598-574719842d4c None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:47.053601 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6800f9df-4469-4c41-9628-e0c2e0ffa3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:47.054629 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800f9df-4469-4c41-9628-e0c2e0ffa3f6 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:47.055160 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800f9df-4469-4c41-9628-e0c2e0ffa3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:47.055697 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800f9df-4469-4c41-9628-e0c2e0ffa3f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:47.057698 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6800f9df-4469-4c41-9628-e0c2e0ffa3f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:47.059412 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6800f9df-4469-4c41-9628-e0c2e0ffa3f6 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:47.061616 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 144/276] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:20:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:47.067460 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2569bc63-453f-4004-a2a0-c9eba075cde2 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:47.085001 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6800f9df-4469-4c41-9628-e0c2e0ffa3f6 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:47.193796 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2569bc63-453f-4004-a2a0-c9eba075cde2 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:47.212161 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6800f9df-4469-4c41-9628-e0c2e0ffa3f6 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:47.229711 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14115a25-75fe-402c-8588-80f22857aad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:47.230503 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14115a25-75fe-402c-8588-80f22857aad2 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:47.230752 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14115a25-75fe-402c-8588-80f22857aad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:47.231023 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14115a25-75fe-402c-8588-80f22857aad2 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:47.231682 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14115a25-75fe-402c-8588-80f22857aad2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:47.233174 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14115a25-75fe-402c-8588-80f22857aad2 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:47.244824 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:47.254618 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531470d-f3b8-479b-96ef-591cd54ddd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:47.255337 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531470d-f3b8-479b-96ef-591cd54ddd50 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:47.255627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531470d-f3b8-479b-96ef-591cd54ddd50 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:47.255898 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531470d-f3b8-479b-96ef-591cd54ddd50 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:47.257981 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 133/277] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 1 16:20:47 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:20:47.258849 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14115a25-75fe-402c-8588-80f22857aad2 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:47.276361 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5531470d-f3b8-479b-96ef-591cd54ddd50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:47.282004 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-5531470d-f3b8-479b-96ef-591cd54ddd50 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:20:47.282004 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:47.282004 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:47.282004 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:47.282004 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:47.282004 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:47.282004 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:47.282004 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:47.282004 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:47.282004 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:47.282004 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:47.282004 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:47.282004 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:47.282004 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:47.282004 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:47.282004 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:47.283236 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:47.283236 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:47.283236 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:47.283236 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:47.283236 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:47.283236 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:47.283236 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:47.283236 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:47.283236 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:47.283236 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:47.283236 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:47.283236 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:47.283236 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:47.283236 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:47.283236 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:47.283236 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:47.284617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:47.284617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:47.284617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:47.284617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:47.284617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:47.284617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:47.284617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:47.284617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:47.284617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:47.284617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:47.284617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:20:47.284617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:20:47.284617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 145/278] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:20:47.284617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511c1c1-27bf-4ecc-815c-650480e8d508 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:47.285330 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511c1c1-27bf-4ecc-815c-650480e8d508 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:47.285330 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511c1c1-27bf-4ecc-815c-650480e8d508 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:47.286634 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 146/279] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:47.289236 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511c1c1-27bf-4ecc-815c-650480e8d508 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:47.330394 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:47.392292 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14115a25-75fe-402c-8588-80f22857aad2 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:47.419756 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511c1c1-27bf-4ecc-815c-650480e8d508 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:47.432656 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba825193-0a2d-4e77-bab5-166433f12103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:47.433503 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba825193-0a2d-4e77-bab5-166433f12103 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:47.433969 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba825193-0a2d-4e77-bab5-166433f12103 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:47.434261 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba825193-0a2d-4e77-bab5-166433f12103 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:47.446737 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba825193-0a2d-4e77-bab5-166433f12103 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:47.453122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-ba825193-0a2d-4e77-bab5-166433f12103 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:20:47.453122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:47.453122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:47.453122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:47.453122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:47.453122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:47.453122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:47.453122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:47.453122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:47.453122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:47.453122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:47.453122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:47.453122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:47.453122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:47.453122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:47.453122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:47.454122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:47.454122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:47.454122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:47.454122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:47.454122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:47.454122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:47.454122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:47.454122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:47.454122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:47.454122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:47.454122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:47.454122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:47.454122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:47.454122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:47.454122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:47.454122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:47.454913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:47.454913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:47.454913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:47.454913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:47.454913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:47.454913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:47.454913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:47.454913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:47.454913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:47.454913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:47.454913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:20:47.454913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:20:47.454913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 134/280] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:47.456333 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f99fc-1831-434e-bbb1-a00de8bdf7f6 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:47.456333 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f99fc-1831-434e-bbb1-a00de8bdf7f6 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:47.456333 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f99fc-1831-434e-bbb1-a00de8bdf7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:47.468442 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63013e3d-105d-44ac-8a5b-b6f5de092b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:47.469153 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63013e3d-105d-44ac-8a5b-b6f5de092b84 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:47.469507 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63013e3d-105d-44ac-8a5b-b6f5de092b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:47.469779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63013e3d-105d-44ac-8a5b-b6f5de092b84 None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:47.470477 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63013e3d-105d-44ac-8a5b-b6f5de092b84 None admin] RBAC: Adding query filter params (name=member) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:47.471406 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63013e3d-105d-44ac-8a5b-b6f5de092b84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:47.472150 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63013e3d-105d-44ac-8a5b-b6f5de092b84 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:47.475128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-1e2f99fc-1831-434e-bbb1-a00de8bdf7f6 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:47.476328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-1e2f99fc-1831-434e-bbb1-a00de8bdf7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:47.484309 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 147/281] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 1 16:20:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:20:47.484309 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63013e3d-105d-44ac-8a5b-b6f5de092b84 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:47.575018 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:47.590545 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 135/282] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:47.591704 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f99fc-1831-434e-bbb1-a00de8bdf7f6 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:47.640930 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:47.643864 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63013e3d-105d-44ac-8a5b-b6f5de092b84 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:47.692605 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6e20f-625f-4986-a66c-0ab587b66d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:47.693557 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6e20f-625f-4986-a66c-0ab587b66d21 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:47.694030 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6e20f-625f-4986-a66c-0ab587b66d21 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:47.694528 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6e20f-625f-4986-a66c-0ab587b66d21 None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:47.695473 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6e20f-625f-4986-a66c-0ab587b66d21 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:47.696041 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6e20f-625f-4986-a66c-0ab587b66d21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:47.697714 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6e20f-625f-4986-a66c-0ab587b66d21 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:47.704783 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f99fc-1831-434e-bbb1-a00de8bdf7f6 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:47.707619 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 148/283] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 1 16:20:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:47.713225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6e20f-625f-4986-a66c-0ab587b66d21 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:47.804264 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6068ad9a-ecd5-4d7e-b337-17127da08dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:47.804921 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068ad9a-ecd5-4d7e-b337-17127da08dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:47.805202 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068ad9a-ecd5-4d7e-b337-17127da08dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:47.805498 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068ad9a-ecd5-4d7e-b337-17127da08dd5 None admin] PATH_INFO: `/v3/users/730af05385b542238f3bac66d126bf87` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:47.826225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6e20f-625f-4986-a66c-0ab587b66d21 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:47.838607 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:47.845392 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6068ad9a-ecd5-4d7e-b337-17127da08dd5 None admin] RBAC: Authorizing `identity:get_user(user_id=730af05385b542238f3bac66d126bf87)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:47.858370 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6068ad9a-ecd5-4d7e-b337-17127da08dd5 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:47.858370 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 136/284] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:47 2022] GET /identity/v3/users/730af05385b542238f3bac66d126bf87 => generated 293 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:47.862076 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6068ad9a-ecd5-4d7e-b337-17127da08dd5 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:47.899082 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867db9d1-9f3a-488a-92df-baa1790b9009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:47.900024 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867db9d1-9f3a-488a-92df-baa1790b9009 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:47.900504 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867db9d1-9f3a-488a-92df-baa1790b9009 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:47.900922 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867db9d1-9f3a-488a-92df-baa1790b9009 None admin] PATH_INFO: `/v3/users/e3edf84518fd4cfd849f110ba2ff7217` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:47.904759 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867db9d1-9f3a-488a-92df-baa1790b9009 None admin] RBAC: Authorizing `identity:get_user(user_id=e3edf84518fd4cfd849f110ba2ff7217)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:47.906463 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867db9d1-9f3a-488a-92df-baa1790b9009 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:47.908242 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 149/285] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:47 2022] GET /identity/v3/users/e3edf84518fd4cfd849f110ba2ff7217 => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:20:47.916595 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867db9d1-9f3a-488a-92df-baa1790b9009 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:48.004016 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6068ad9a-ecd5-4d7e-b337-17127da08dd5 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:48.005862 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:48.041131 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4012a9-c968-411e-a73f-8ae07c4b3fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:48.041538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4012a9-c968-411e-a73f-8ae07c4b3fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:48.042482 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4012a9-c968-411e-a73f-8ae07c4b3fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:48.042482 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4012a9-c968-411e-a73f-8ae07c4b3fd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:48.043588 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4012a9-c968-411e-a73f-8ae07c4b3fd0 None admin] RBAC: Adding query filter params (role.id=45545334e1fc473c9d85fc91a9b0a35c, user.id=730af05385b542238f3bac66d126bf87, scope.system=all) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:48.043588 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4012a9-c968-411e-a73f-8ae07c4b3fd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:48.045716 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4012a9-c968-411e-a73f-8ae07c4b3fd0 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:48.052507 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 137/286] 10.222.0.18 () {56 vars in 1308 bytes} [Thu Sep 1 16:20:47 2022] GET /identity/v3/role_assignments?user.id=730af05385b542238f3bac66d126bf87&scope.system=all&role.id=45545334e1fc473c9d85fc91a9b0a35c => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:48.053488 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4012a9-c968-411e-a73f-8ae07c4b3fd0 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:48.120325 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867db9d1-9f3a-488a-92df-baa1790b9009 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:48.213196 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564207e-b2ed-4bf5-9f21-e24d31793514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:48.213871 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564207e-b2ed-4bf5-9f21-e24d31793514 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:48.214217 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564207e-b2ed-4bf5-9f21-e24d31793514 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:48.214560 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564207e-b2ed-4bf5-9f21-e24d31793514 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:48.226553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4564207e-b2ed-4bf5-9f21-e24d31793514 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:48.232974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-4564207e-b2ed-4bf5-9f21-e24d31793514 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:20:48.232974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:48.232974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:48.232974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:48.232974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:48.232974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:48.232974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:48.232974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:48.232974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:48.232974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:48.232974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:48.232974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:48.232974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:48.232974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:48.232974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:48.232974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:48.233822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:48.233822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:48.233822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:48.233822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:48.233822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:48.233822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:48.233822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:48.233822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:48.233822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:48.233822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:48.233822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:48.233822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:48.233822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:48.233822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:48.233822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:48.233822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:48.234637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:48.234637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:48.234637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:48.234637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:48.234637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:48.234637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:48.234637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:48.234637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:48.234637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:48.234637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:48.234637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:20:48.234637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:20:48.234637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 150/287] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:48.240725 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564207e-b2ed-4bf5-9f21-e24d31793514 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:48.243707 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4012a9-c968-411e-a73f-8ae07c4b3fd0 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:48.310629 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:48.312148 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6803e-a8fb-4e68-aa0f-adb662814f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:48.314461 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6803e-a8fb-4e68-aa0f-adb662814f69 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:48.314461 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6803e-a8fb-4e68-aa0f-adb662814f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:48.314461 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6803e-a8fb-4e68-aa0f-adb662814f69 None admin] PATH_INFO: `/v3/projects/cc5b99bf5631484eb25353883854e856` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:48.315792 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6803e-a8fb-4e68-aa0f-adb662814f69 None admin] RBAC: Authorizing `identity:get_project(project_id=cc5b99bf5631484eb25353883854e856)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:48.317219 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6803e-a8fb-4e68-aa0f-adb662814f69 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:48.318898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 138/288] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:48 2022] GET /identity/v3/projects/cc5b99bf5631484eb25353883854e856 => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:48.325391 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6803e-a8fb-4e68-aa0f-adb662814f69 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:48.390751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:48.436038 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564207e-b2ed-4bf5-9f21-e24d31793514 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:48.506931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a29d67-bdf2-4fab-af5d-f619237d0ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:48.507550 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a29d67-bdf2-4fab-af5d-f619237d0ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:48.507833 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a29d67-bdf2-4fab-af5d-f619237d0ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:48.508108 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a29d67-bdf2-4fab-af5d-f619237d0ed1 None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:48.508841 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a29d67-bdf2-4fab-af5d-f619237d0ed1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:48.509149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a29d67-bdf2-4fab-af5d-f619237d0ed1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:48.511325 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a29d67-bdf2-4fab-af5d-f619237d0ed1 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:48.525026 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6803e-a8fb-4e68-aa0f-adb662814f69 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:48.526793 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 151/289] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 1 16:20:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:48.533820 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a29d67-bdf2-4fab-af5d-f619237d0ed1 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:48.592213 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544f07f-f277-4cfe-ad4b-03110f56bc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:48.592841 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544f07f-f277-4cfe-ad4b-03110f56bc10 None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:48.593124 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544f07f-f277-4cfe-ad4b-03110f56bc10 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:48.593400 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544f07f-f277-4cfe-ad4b-03110f56bc10 None admin] PATH_INFO: `/v3/projects/cc5b99bf5631484eb25353883854e856/users/e3edf84518fd4cfd849f110ba2ff7217/roles/12fe0aa7f1174edfb109315b88d6267b` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:48.598942 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1544f07f-f277-4cfe-ad4b-03110f56bc10 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc5b99bf5631484eb25353883854e856, user_id=e3edf84518fd4cfd849f110ba2ff7217, role_id=12fe0aa7f1174edfb109315b88d6267b)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:48.604103 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1544f07f-f277-4cfe-ad4b-03110f56bc10 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:48.627681 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 139/290] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 1 16:20:48 2022] PUT /identity/v3/projects/cc5b99bf5631484eb25353883854e856/users/e3edf84518fd4cfd849f110ba2ff7217/roles/12fe0aa7f1174edfb109315b88d6267b => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:20:48.684015 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:48.688253 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a29d67-bdf2-4fab-af5d-f619237d0ed1 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:48.739457 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5e04b-6d78-41ee-afa6-1b147161d5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:48.740114 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5e04b-6d78-41ee-afa6-1b147161d5dd None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:48.740395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5e04b-6d78-41ee-afa6-1b147161d5dd None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:48.740668 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5e04b-6d78-41ee-afa6-1b147161d5dd None admin] PATH_INFO: `/v3/users/aac849f048fe412ea0e49d54985b6c6e` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:48.741900 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5e04b-6d78-41ee-afa6-1b147161d5dd None admin] RBAC: Authorizing `identity:get_user(user_id=aac849f048fe412ea0e49d54985b6c6e)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:48.743545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5e04b-6d78-41ee-afa6-1b147161d5dd None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:48.745561 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 152/291] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:48 2022] GET /identity/v3/users/aac849f048fe412ea0e49d54985b6c6e => generated 297 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:48.751919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544f07f-f277-4cfe-ad4b-03110f56bc10 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:48.878752 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:48.889500 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544f07f-f277-4cfe-ad4b-03110f56bc10 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:48.937314 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfe4b84-92b7-4e71-b78a-2f627d66f714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:48.937931 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe4b84-92b7-4e71-b78a-2f627d66f714 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:48.938213 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe4b84-92b7-4e71-b78a-2f627d66f714 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:48.938517 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe4b84-92b7-4e71-b78a-2f627d66f714 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:49.025641 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfe4b84-92b7-4e71-b78a-2f627d66f714 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:49.027201 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfe4b84-92b7-4e71-b78a-2f627d66f714 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:49.144477 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:49.155877 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 140/292] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:20:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:20:49.161347 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5e04b-6d78-41ee-afa6-1b147161d5dd None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:49.283654 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5e04b-6d78-41ee-afa6-1b147161d5dd None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:49.328659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cdbbcd-ec30-4375-98f0-6bba44e12473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:49.329434 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cdbbcd-ec30-4375-98f0-6bba44e12473 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:49.329556 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cdbbcd-ec30-4375-98f0-6bba44e12473 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:49.329837 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cdbbcd-ec30-4375-98f0-6bba44e12473 None admin] PATH_INFO: `/v3/projects/cc5b99bf5631484eb25353883854e856` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:49.330970 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cdbbcd-ec30-4375-98f0-6bba44e12473 None admin] RBAC: Authorizing `identity:get_project(project_id=cc5b99bf5631484eb25353883854e856)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:49.332431 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cdbbcd-ec30-4375-98f0-6bba44e12473 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:49.334253 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 153/293] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:49 2022] GET /identity/v3/projects/cc5b99bf5631484eb25353883854e856 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:49.340023 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfe4b84-92b7-4e71-b78a-2f627d66f714 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:49.497119 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfe4b84-92b7-4e71-b78a-2f627d66f714 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:49.541086 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db78aae1-e922-431d-9728-97a900b18f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:49.541702 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78aae1-e922-431d-9728-97a900b18f1d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:49.541985 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78aae1-e922-431d-9728-97a900b18f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:49.542297 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78aae1-e922-431d-9728-97a900b18f1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:49.543025 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db78aae1-e922-431d-9728-97a900b18f1d None admin] RBAC: Adding query filter params (role.id=45545334e1fc473c9d85fc91a9b0a35c, scope.project.id=cc5b99bf5631484eb25353883854e856, user.id=aac849f048fe412ea0e49d54985b6c6e) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:49.543332 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db78aae1-e922-431d-9728-97a900b18f1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:49.544878 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db78aae1-e922-431d-9728-97a900b18f1d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:49.551075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 141/294] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:20:49 2022] GET /identity/v3/role_assignments?user.id=aac849f048fe412ea0e49d54985b6c6e&scope.project.id=cc5b99bf5631484eb25353883854e856&role.id=45545334e1fc473c9d85fc91a9b0a35c => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:49.604154 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953143b-3033-42bd-b0cf-762952b19920 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:49.604502 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953143b-3033-42bd-b0cf-762952b19920 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:49.604794 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953143b-3033-42bd-b0cf-762952b19920 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:49.606498 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 154/295] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:49.612061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4461ac0-c47e-4559-82cd-816dd8ca0ae4 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:49.612357 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4461ac0-c47e-4559-82cd-816dd8ca0ae4 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:49.612633 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4461ac0-c47e-4559-82cd-816dd8ca0ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:49.632411 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-d4461ac0-c47e-4559-82cd-816dd8ca0ae4 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:49.633662 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d4461ac0-c47e-4559-82cd-816dd8ca0ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:49.707538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:49.730003 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 142/296] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:49.789148 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a2424-a2ca-496f-a54e-51d4c72850c8 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:49.789446 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a2424-a2ca-496f-a54e-51d4c72850c8 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:49.789706 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a2424-a2ca-496f-a54e-51d4c72850c8 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:49.791346 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 155/297] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:49.796576 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f415e1-7b37-4466-beb2-8e110321a1ff None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:49.796871 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f415e1-7b37-4466-beb2-8e110321a1ff None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:49.797151 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f415e1-7b37-4466-beb2-8e110321a1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:49.822707 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-29f415e1-7b37-4466-beb2-8e110321a1ff None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:49.824000 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-29f415e1-7b37-4466-beb2-8e110321a1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:49.900551 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 143/298] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:50.081322 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a2424-a2ca-496f-a54e-51d4c72850c8 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:50.126854 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:50.200572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a2424-a2ca-496f-a54e-51d4c72850c8 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:50.224737 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f415e1-7b37-4466-beb2-8e110321a1ff None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:50.250280 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70bf5b-5e8f-4d66-849c-5757f1fccf82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:50.250930 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70bf5b-5e8f-4d66-849c-5757f1fccf82 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:50.251214 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70bf5b-5e8f-4d66-849c-5757f1fccf82 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:50.251490 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70bf5b-5e8f-4d66-849c-5757f1fccf82 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:50.262700 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f70bf5b-5e8f-4d66-849c-5757f1fccf82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:50.267911 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-7f70bf5b-5e8f-4d66-849c-5757f1fccf82 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:20:50.267911 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:50.267911 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:50.267911 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:50.267911 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:50.267911 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:50.267911 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:50.267911 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:50.267911 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:50.267911 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:50.267911 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:50.267911 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:50.267911 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:50.267911 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:50.267911 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:50.267911 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:50.268791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:50.268791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:50.268791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:50.268791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:50.268791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:50.268791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:50.268791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:50.268791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:50.268791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:50.268791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:50.268791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:50.268791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:50.268791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:50.268791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:50.268791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:50.268791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:50.269549 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:50.269549 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:50.269549 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:50.269549 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:50.269549 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:50.269549 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:50.269549 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:50.269549 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:50.269549 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:50.269549 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:50.269549 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:20:50.269549 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:20:50.269549 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 156/299] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:50.275910 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70bf5b-5e8f-4d66-849c-5757f1fccf82 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:50.343213 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f415e1-7b37-4466-beb2-8e110321a1ff None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:50.364120 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:50.380329 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:50.386389 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec09310-922f-4c2e-9b79-75b50249a7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:50.386519 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70bf5b-5e8f-4d66-849c-5757f1fccf82 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:50.387038 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec09310-922f-4c2e-9b79-75b50249a7a2 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:50.387320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec09310-922f-4c2e-9b79-75b50249a7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:50.387593 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec09310-922f-4c2e-9b79-75b50249a7a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:50.388714 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec09310-922f-4c2e-9b79-75b50249a7a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:50.390156 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec09310-922f-4c2e-9b79-75b50249a7a2 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:50.391893 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 144/300] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:20:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:50.392856 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66c8ab-3900-4555-8b3a-7e805883dd41 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:50.393141 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66c8ab-3900-4555-8b3a-7e805883dd41 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:50.393417 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66c8ab-3900-4555-8b3a-7e805883dd41 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:50.394746 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 145/301] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:50.399310 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef66c8ab-3900-4555-8b3a-7e805883dd41 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:50.428781 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf75ac-63b7-4eae-8f88-59ee7e1dd9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:50.429399 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf75ac-63b7-4eae-8f88-59ee7e1dd9b0 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:50.429671 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf75ac-63b7-4eae-8f88-59ee7e1dd9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:50.429942 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf75ac-63b7-4eae-8f88-59ee7e1dd9b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:50.430626 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf75ac-63b7-4eae-8f88-59ee7e1dd9b0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:50.430928 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf75ac-63b7-4eae-8f88-59ee7e1dd9b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:50.432296 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf75ac-63b7-4eae-8f88-59ee7e1dd9b0 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:50.440434 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 157/302] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 1 16:20:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:20:50.440434 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d711be5-f376-4e54-a38c-731ff6eb4ed2 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:50.440434 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d711be5-f376-4e54-a38c-731ff6eb4ed2 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:50.440434 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d711be5-f376-4e54-a38c-731ff6eb4ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:50.461644 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-6d711be5-f376-4e54-a38c-731ff6eb4ed2 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:50.462835 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-6d711be5-f376-4e54-a38c-731ff6eb4ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:50.511794 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef66c8ab-3900-4555-8b3a-7e805883dd41 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:50.542331 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 158/303] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:50.542331 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d711be5-f376-4e54-a38c-731ff6eb4ed2 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:50.554612 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc1d21-f8ef-4263-a189-2ca4628ba0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:50.555435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc1d21-f8ef-4263-a189-2ca4628ba0eb None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:50.555845 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc1d21-f8ef-4263-a189-2ca4628ba0eb None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:50.556257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc1d21-f8ef-4263-a189-2ca4628ba0eb None admin] PATH_INFO: `/v3/groups` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:50.557024 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc1d21-f8ef-4263-a189-2ca4628ba0eb None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:50.558894 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc1d21-f8ef-4263-a189-2ca4628ba0eb None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:50.579704 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 146/304] 10.222.0.18 () {60 vars in 1134 bytes} [Thu Sep 1 16:20:50 2022] POST /identity/v3/groups => generated 229 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:20:50.649213 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:50.664837 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d711be5-f376-4e54-a38c-731ff6eb4ed2 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:50.702943 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb29987-3ea5-4365-b946-e6b1b5f4ba94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:50.703912 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb29987-3ea5-4365-b946-e6b1b5f4ba94 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:50.704332 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb29987-3ea5-4365-b946-e6b1b5f4ba94 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:50.704754 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb29987-3ea5-4365-b946-e6b1b5f4ba94 None admin] PATH_INFO: `/v3/users/395e6a825d0d4402853bc35b9f656688` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:50.720895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb29987-3ea5-4365-b946-e6b1b5f4ba94 None admin] RBAC: Authorizing `identity:get_user(user_id=395e6a825d0d4402853bc35b9f656688)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:50.722656 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb29987-3ea5-4365-b946-e6b1b5f4ba94 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:50.724548 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 159/305] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:50 2022] GET /identity/v3/users/395e6a825d0d4402853bc35b9f656688 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:50.730805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc1d21-f8ef-4263-a189-2ca4628ba0eb None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:50.781911 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:50.811305 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb29987-3ea5-4365-b946-e6b1b5f4ba94 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:50.856382 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc1d21-f8ef-4263-a189-2ca4628ba0eb None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:50.903209 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f5b47-6d60-4f93-be01-cc0b338ec6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:50.904746 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f5b47-6d60-4f93-be01-cc0b338ec6f9 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:50.905210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f5b47-6d60-4f93-be01-cc0b338ec6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:50.905662 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f5b47-6d60-4f93-be01-cc0b338ec6f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:50.906616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f5b47-6d60-4f93-be01-cc0b338ec6f9 None admin] RBAC: Adding query filter params (role.id=12fe0aa7f1174edfb109315b88d6267b, user.id=395e6a825d0d4402853bc35b9f656688, scope.system=all) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:50.907134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f5b47-6d60-4f93-be01-cc0b338ec6f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:50.909285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f5b47-6d60-4f93-be01-cc0b338ec6f9 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:50.922266 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 147/306] 10.222.0.18 () {56 vars in 1308 bytes} [Thu Sep 1 16:20:50 2022] GET /identity/v3/role_assignments?user.id=395e6a825d0d4402853bc35b9f656688&scope.system=all&role.id=12fe0aa7f1174edfb109315b88d6267b => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:50.942645 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb29987-3ea5-4365-b946-e6b1b5f4ba94 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:50.981635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36871499-d046-4bc2-b098-5db5f598a296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:50.982436 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36871499-d046-4bc2-b098-5db5f598a296 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:50.982720 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36871499-d046-4bc2-b098-5db5f598a296 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:50.983000 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36871499-d046-4bc2-b098-5db5f598a296 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:50.994922 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36871499-d046-4bc2-b098-5db5f598a296 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:51.000704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-36871499-d046-4bc2-b098-5db5f598a296 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:20:51.000704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:51.000704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:51.000704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:51.000704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:51.000704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:51.000704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:51.000704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:51.000704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:51.000704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:51.000704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:51.000704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:51.000704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:51.000704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:51.000704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:51.000704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:51.001640 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:51.001640 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:51.001640 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:51.001640 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:51.001640 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:51.001640 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:51.001640 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:51.001640 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:51.001640 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:51.001640 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:51.001640 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:51.001640 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:51.001640 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:51.001640 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:51.001640 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:51.001640 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:51.002922 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:51.002922 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:51.002922 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:51.002922 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:51.002922 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:51.002922 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:51.002922 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:51.002922 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:51.002922 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:51.002922 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:51.002922 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:20:51.002922 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:20:51.002922 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 160/307] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:20:51.008143 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f5b47-6d60-4f93-be01-cc0b338ec6f9 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:51.068425 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262636a4-c461-4d15-b97c-b04327f78462 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:51.068740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262636a4-c461-4d15-b97c-b04327f78462 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:51.071632 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262636a4-c461-4d15-b97c-b04327f78462 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:51.071632 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 161/308] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:20:51.077921 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bb550-43ac-4b1f-9abd-4163200a445e None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:51.078209 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bb550-43ac-4b1f-9abd-4163200a445e None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:51.078547 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bb550-43ac-4b1f-9abd-4163200a445e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:51.094647 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:51.100607 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-b57bb550-43ac-4b1f-9abd-4163200a445e None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:51.102974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-b57bb550-43ac-4b1f-9abd-4163200a445e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:51.132825 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f5b47-6d60-4f93-be01-cc0b338ec6f9 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:51.148023 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:51.181973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d355fe4-dbb5-48d3-a71c-57aece07f245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:51.182789 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d355fe4-dbb5-48d3-a71c-57aece07f245 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:51.183073 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d355fe4-dbb5-48d3-a71c-57aece07f245 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:51.183576 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d355fe4-dbb5-48d3-a71c-57aece07f245 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:51.184637 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d355fe4-dbb5-48d3-a71c-57aece07f245 None admin] RBAC: Adding query filter params (name=member) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:51.184948 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d355fe4-dbb5-48d3-a71c-57aece07f245 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:51.187181 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d355fe4-dbb5-48d3-a71c-57aece07f245 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:51.194202 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 162/309] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 01 16:20:51.196133 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 148/310] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 1 16:20:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:51.201824 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57bb550-43ac-4b1f-9abd-4163200a445e None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:51.305339 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57bb550-43ac-4b1f-9abd-4163200a445e None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:51.341207 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cca8f4-0610-44cf-bfb8-637501f42d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:51.341925 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cca8f4-0610-44cf-bfb8-637501f42d32 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:51.342208 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cca8f4-0610-44cf-bfb8-637501f42d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:51.342505 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cca8f4-0610-44cf-bfb8-637501f42d32 None admin] PATH_INFO: `/v3/users/730af05385b542238f3bac66d126bf87` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:51.343624 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cca8f4-0610-44cf-bfb8-637501f42d32 None admin] RBAC: Authorizing `identity:get_user(user_id=730af05385b542238f3bac66d126bf87)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:51.345055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cca8f4-0610-44cf-bfb8-637501f42d32 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:51.352027 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 163/311] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:51 2022] GET /identity/v3/users/730af05385b542238f3bac66d126bf87 => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:51.359298 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d355fe4-dbb5-48d3-a71c-57aece07f245 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:51.455303 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cca8f4-0610-44cf-bfb8-637501f42d32 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:51.459297 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:51.476750 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d355fe4-dbb5-48d3-a71c-57aece07f245 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:51.512125 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e318d3-cc35-4f3f-a679-f4bae61a6465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:51.513567 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e318d3-cc35-4f3f-a679-f4bae61a6465 None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:51.514047 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e318d3-cc35-4f3f-a679-f4bae61a6465 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:51.514633 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e318d3-cc35-4f3f-a679-f4bae61a6465 None admin] PATH_INFO: `/v3/system/users/730af05385b542238f3bac66d126bf87/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:51.516282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e318d3-cc35-4f3f-a679-f4bae61a6465 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=730af05385b542238f3bac66d126bf87, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:51.518045 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e318d3-cc35-4f3f-a679-f4bae61a6465 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:51.528244 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 149/312] 10.222.0.18 () {58 vars in 1334 bytes} [Thu Sep 1 16:20:51 2022] PUT /identity/v3/system/users/730af05385b542238f3bac66d126bf87/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:20:51.576474 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cca8f4-0610-44cf-bfb8-637501f42d32 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:51.619113 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee89b877-038e-4640-a5ea-bcf06d4a3c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:51.619769 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89b877-038e-4640-a5ea-bcf06d4a3c91 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:51.620050 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89b877-038e-4640-a5ea-bcf06d4a3c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:51.620323 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89b877-038e-4640-a5ea-bcf06d4a3c91 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:51.633341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee89b877-038e-4640-a5ea-bcf06d4a3c91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:51.639053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-ee89b877-038e-4640-a5ea-bcf06d4a3c91 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:20:51.639053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:51.639053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:51.639053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:51.639053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:51.639053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:51.639053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:51.639053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:51.639053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:51.639053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:51.639053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:51.639053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:51.639053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:51.639053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:51.639053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:51.639053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:51.640023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:51.640023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:51.640023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:51.640023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:51.640023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:51.640023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:51.640023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:51.640023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:51.640023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:51.640023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:51.640023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:51.640023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:51.640023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:51.640023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:51.640023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:51.640023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:51.640801 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:51.640801 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:51.640801 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:51.640801 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:51.640801 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:51.640801 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:51.640801 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:51.640801 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:51.640801 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:51.640801 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:51.640801 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:20:51.640801 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:20:51.640801 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 164/313] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:51.646594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e318d3-cc35-4f3f-a679-f4bae61a6465 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:51.689486 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:51.762525 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e318d3-cc35-4f3f-a679-f4bae61a6465 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:51.800602 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9a0e5-9a0a-4b7f-8169-7071e6cb9ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:51.801224 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9a0e5-9a0a-4b7f-8169-7071e6cb9ddf None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:51.801510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9a0e5-9a0a-4b7f-8169-7071e6cb9ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:51.801785 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9a0e5-9a0a-4b7f-8169-7071e6cb9ddf None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:51.802490 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a9a0e5-9a0a-4b7f-8169-7071e6cb9ddf None admin] RBAC: Adding query filter params (name=reader) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:51.802800 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a9a0e5-9a0a-4b7f-8169-7071e6cb9ddf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:51.804167 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a9a0e5-9a0a-4b7f-8169-7071e6cb9ddf None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:51.813023 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 150/314] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 1 16:20:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:51.818590 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee89b877-038e-4640-a5ea-bcf06d4a3c91 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:51.884883 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:51.924921 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee89b877-038e-4640-a5ea-bcf06d4a3c91 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:51.958398 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23486a8-ec17-460b-aba9-6cdf4fc0c750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:51.959093 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23486a8-ec17-460b-aba9-6cdf4fc0c750 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:51.959379 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23486a8-ec17-460b-aba9-6cdf4fc0c750 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:51.959651 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23486a8-ec17-460b-aba9-6cdf4fc0c750 None admin] PATH_INFO: `/v3/users/e3edf84518fd4cfd849f110ba2ff7217` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:51.960779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23486a8-ec17-460b-aba9-6cdf4fc0c750 None admin] RBAC: Authorizing `identity:get_user(user_id=e3edf84518fd4cfd849f110ba2ff7217)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:51.962231 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23486a8-ec17-460b-aba9-6cdf4fc0c750 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:51.963835 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 165/315] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:51 2022] GET /identity/v3/users/e3edf84518fd4cfd849f110ba2ff7217 => generated 297 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:51.970054 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9a0e5-9a0a-4b7f-8169-7071e6cb9ddf None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:52.071486 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:52.077722 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9a0e5-9a0a-4b7f-8169-7071e6cb9ddf None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:52.115661 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c31c09-f870-4496-be29-735255a0406c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:52.116261 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c31c09-f870-4496-be29-735255a0406c None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:52.116551 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c31c09-f870-4496-be29-735255a0406c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:52.116829 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c31c09-f870-4496-be29-735255a0406c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:52.194518 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c31c09-f870-4496-be29-735255a0406c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:52.195992 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c31c09-f870-4496-be29-735255a0406c None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:52.291717 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:52.298804 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 151/316] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:20:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:20:52.304197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23486a8-ec17-460b-aba9-6cdf4fc0c750 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:52.408893 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23486a8-ec17-460b-aba9-6cdf4fc0c750 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:52.439998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:52.445126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5829f53-82ed-4e78-8792-2a27fdfd931f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:52.445826 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5829f53-82ed-4e78-8792-2a27fdfd931f None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:52.446113 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5829f53-82ed-4e78-8792-2a27fdfd931f None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:52.446501 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5829f53-82ed-4e78-8792-2a27fdfd931f None admin] PATH_INFO: `/v3/projects/cc5b99bf5631484eb25353883854e856` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:52.447638 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5829f53-82ed-4e78-8792-2a27fdfd931f None admin] RBAC: Authorizing `identity:get_project(project_id=cc5b99bf5631484eb25353883854e856)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:52.449218 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5829f53-82ed-4e78-8792-2a27fdfd931f None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:52.452439 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 166/317] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:52 2022] GET /identity/v3/projects/cc5b99bf5631484eb25353883854e856 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:52.458450 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c31c09-f870-4496-be29-735255a0406c None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:52.562278 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c31c09-f870-4496-be29-735255a0406c None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:52.598707 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f51e7-c7cb-47db-843e-b2103cb9bda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:52.599683 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f51e7-c7cb-47db-843e-b2103cb9bda3 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:52.600165 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f51e7-c7cb-47db-843e-b2103cb9bda3 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:52.600635 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f51e7-c7cb-47db-843e-b2103cb9bda3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:52.601573 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f51e7-c7cb-47db-843e-b2103cb9bda3 None admin] RBAC: Adding query filter params (role.id=12fe0aa7f1174edfb109315b88d6267b, scope.project.id=cc5b99bf5631484eb25353883854e856, user.id=e3edf84518fd4cfd849f110ba2ff7217) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:52.602097 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f51e7-c7cb-47db-843e-b2103cb9bda3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:52.603760 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f51e7-c7cb-47db-843e-b2103cb9bda3 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:52.610573 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 152/318] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:20:52 2022] GET /identity/v3/role_assignments?user.id=e3edf84518fd4cfd849f110ba2ff7217&scope.project.id=cc5b99bf5631484eb25353883854e856&role.id=12fe0aa7f1174edfb109315b88d6267b => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:52.631817 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f9360-cfd7-4498-9d3b-edb7fb69bab6 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:52.632307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f9360-cfd7-4498-9d3b-edb7fb69bab6 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:52.632307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f9360-cfd7-4498-9d3b-edb7fb69bab6 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:52.633936 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 167/319] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:52.640129 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bb706-78aa-4e8b-bcdd-d805ff91f95b None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:52.640424 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bb706-78aa-4e8b-bcdd-d805ff91f95b None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:52.640686 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bb706-78aa-4e8b-bcdd-d805ff91f95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:52.663036 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-d56bb706-78aa-4e8b-bcdd-d805ff91f95b None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:52.664316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d56bb706-78aa-4e8b-bcdd-d805ff91f95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:52.726846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:52.738732 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 153/320] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:52.977495 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aadab18-ea1e-4dad-b9e3-9c5a05d081c7 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:52.977751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aadab18-ea1e-4dad-b9e3-9c5a05d081c7 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:52.977954 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aadab18-ea1e-4dad-b9e3-9c5a05d081c7 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:52.979463 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 168/321] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:52.985274 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a2b30-b048-46ed-b1d2-a68ef9fb1de7 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:52.985568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a2b30-b048-46ed-b1d2-a68ef9fb1de7 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:52.985846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a2b30-b048-46ed-b1d2-a68ef9fb1de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:53.004833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-9c2a2b30-b048-46ed-b1d2-a68ef9fb1de7 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:53.005643 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aadab18-ea1e-4dad-b9e3-9c5a05d081c7 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:53.006036 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9c2a2b30-b048-46ed-b1d2-a68ef9fb1de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:53.075616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 154/322] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:53.108889 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aadab18-ea1e-4dad-b9e3-9c5a05d081c7 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:53.142835 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d5f77a-0ea1-417f-8d2b-0290d68141f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:53.143468 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5f77a-0ea1-417f-8d2b-0290d68141f1 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:53.143750 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5f77a-0ea1-417f-8d2b-0290d68141f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:53.144025 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5f77a-0ea1-417f-8d2b-0290d68141f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:53.145121 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d5f77a-0ea1-417f-8d2b-0290d68141f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:53.146582 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d5f77a-0ea1-417f-8d2b-0290d68141f1 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:53.148328 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 169/323] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:20:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:53.153705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a2b30-b048-46ed-b1d2-a68ef9fb1de7 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:53.252334 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a2b30-b048-46ed-b1d2-a68ef9fb1de7 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:53.272027 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:53.287305 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b877a-b550-4ffd-8447-68903cb473e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:53.287927 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b877a-b550-4ffd-8447-68903cb473e1 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:53.288207 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b877a-b550-4ffd-8447-68903cb473e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:53.288480 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b877a-b550-4ffd-8447-68903cb473e1 None admin] PATH_INFO: `/v3/groups` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:53.289118 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b877a-b550-4ffd-8447-68903cb473e1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:53.290577 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b877a-b550-4ffd-8447-68903cb473e1 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:53.307243 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 155/324] 10.222.0.18 () {60 vars in 1134 bytes} [Thu Sep 1 16:20:53 2022] POST /identity/v3/groups => generated 226 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:20:53.343546 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d5f77a-0ea1-417f-8d2b-0290d68141f1 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:53.423323 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:53.459621 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d5f77a-0ea1-417f-8d2b-0290d68141f1 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:53.499503 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf655f-19c4-4f4d-8774-2cf349d62fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:53.500151 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf655f-19c4-4f4d-8774-2cf349d62fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:53.500438 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf655f-19c4-4f4d-8774-2cf349d62fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:53.500715 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf655f-19c4-4f4d-8774-2cf349d62fd7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:53.511801 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcf655f-19c4-4f4d-8774-2cf349d62fd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:53.517560 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-fbcf655f-19c4-4f4d-8774-2cf349d62fd7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:20:53.517560 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:53.517560 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:53.517560 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:53.517560 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:53.517560 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:53.517560 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:53.517560 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:53.517560 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:53.517560 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:53.517560 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:53.517560 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:53.517560 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:53.517560 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:53.517560 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:53.517560 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:53.518615 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:53.518615 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:53.518615 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:53.518615 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:53.518615 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:53.518615 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:53.518615 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:53.518615 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:53.518615 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:53.518615 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:53.518615 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:53.518615 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:53.518615 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:53.518615 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:53.518615 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:53.518615 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:53.519938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:53.519938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:53.519938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:53.519938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:53.519938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:53.519938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:53.519938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:53.519938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:53.519938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:53.519938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:53.519938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:20:53.519938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:20:53.519938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 170/325] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:53.520763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da724db2-2651-4bff-9aac-09a42437f350 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:53.521066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da724db2-2651-4bff-9aac-09a42437f350 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:53.521346 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da724db2-2651-4bff-9aac-09a42437f350 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:53.523415 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 156/326] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:53.524383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf655f-19c4-4f4d-8774-2cf349d62fd7 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:53.529717 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3bca2-6123-4bb7-95c4-5f66887e0a73 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:53.530329 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3bca2-6123-4bb7-95c4-5f66887e0a73 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:53.530329 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3bca2-6123-4bb7-95c4-5f66887e0a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:53.555635 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-78b3bca2-6123-4bb7-95c4-5f66887e0a73 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:53.557658 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-78b3bca2-6123-4bb7-95c4-5f66887e0a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:53.639983 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 157/327] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:53.650289 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf655f-19c4-4f4d-8774-2cf349d62fd7 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:53.677581 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:53.684924 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877805f-86dd-47be-8235-4371b2683d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:53.685620 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877805f-86dd-47be-8235-4371b2683d92 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:53.686329 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877805f-86dd-47be-8235-4371b2683d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:53.686618 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877805f-86dd-47be-8235-4371b2683d92 None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:53.687297 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877805f-86dd-47be-8235-4371b2683d92 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:53.687846 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877805f-86dd-47be-8235-4371b2683d92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:53.689599 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877805f-86dd-47be-8235-4371b2683d92 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:53.699212 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 171/328] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 1 16:20:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:53.705429 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3bca2-6123-4bb7-95c4-5f66887e0a73 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:53.816326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3bca2-6123-4bb7-95c4-5f66887e0a73 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:53.833210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:53.859687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881caa49-5061-491f-a0e8-f8a953bc263b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:53.860341 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881caa49-5061-491f-a0e8-f8a953bc263b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:53.860626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881caa49-5061-491f-a0e8-f8a953bc263b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:53.860971 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881caa49-5061-491f-a0e8-f8a953bc263b None admin] PATH_INFO: `/v3/users/395e6a825d0d4402853bc35b9f656688` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:53.862182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881caa49-5061-491f-a0e8-f8a953bc263b None admin] RBAC: Authorizing `identity:get_user(user_id=395e6a825d0d4402853bc35b9f656688)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:53.863686 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881caa49-5061-491f-a0e8-f8a953bc263b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:53.865595 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 158/329] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:53 2022] GET /identity/v3/users/395e6a825d0d4402853bc35b9f656688 => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:20:53.871114 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877805f-86dd-47be-8235-4371b2683d92 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:53.930876 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881caa49-5061-491f-a0e8-f8a953bc263b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:54.008279 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877805f-86dd-47be-8235-4371b2683d92 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:54.056216 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64dbf6-21af-4245-8e93-ef540b8701b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:54.057111 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64dbf6-21af-4245-8e93-ef540b8701b5 None admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:54.057409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64dbf6-21af-4245-8e93-ef540b8701b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:54.057688 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64dbf6-21af-4245-8e93-ef540b8701b5 None admin] PATH_INFO: `/v3/system/users/395e6a825d0d4402853bc35b9f656688/roles/12fe0aa7f1174edfb109315b88d6267b` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:54.059876 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b64dbf6-21af-4245-8e93-ef540b8701b5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=395e6a825d0d4402853bc35b9f656688, role_id=12fe0aa7f1174edfb109315b88d6267b)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:54.061894 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b64dbf6-21af-4245-8e93-ef540b8701b5 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:54.075458 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 172/330] 10.222.0.18 () {58 vars in 1334 bytes} [Thu Sep 1 16:20:53 2022] PUT /identity/v3/system/users/395e6a825d0d4402853bc35b9f656688/roles/12fe0aa7f1174edfb109315b88d6267b => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:20:54.092564 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881caa49-5061-491f-a0e8-f8a953bc263b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:54.133752 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f4b35-865b-4723-bc3a-d129c3f9bac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:54.134774 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f4b35-865b-4723-bc3a-d129c3f9bac1 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:54.135218 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f4b35-865b-4723-bc3a-d129c3f9bac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:54.135620 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f4b35-865b-4723-bc3a-d129c3f9bac1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:54.149602 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9f4b35-865b-4723-bc3a-d129c3f9bac1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:54.155124 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-9b9f4b35-865b-4723-bc3a-d129c3f9bac1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:20:54.155124 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:54.155124 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:54.155124 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:54.155124 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:54.155124 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:54.155124 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:54.155124 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:54.155124 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:54.155124 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:54.155124 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:54.155124 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:54.155124 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:54.155124 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:54.155124 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:54.155124 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:54.156196 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:54.156196 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:54.156196 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:54.156196 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:54.156196 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:54.156196 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:54.156196 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:54.156196 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:54.156196 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:54.156196 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:54.156196 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:54.156196 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:54.156196 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:54.156196 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:54.156196 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:54.156196 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:54.157172 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:54.157172 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:54.157172 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:54.157172 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:54.157172 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:54.157172 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:54.157172 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:54.157172 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:54.157172 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:54.157172 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:54.157172 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:20:54.157172 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:20:54.159143 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 159/331] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:54.165350 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64dbf6-21af-4245-8e93-ef540b8701b5 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:54.236815 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:54.297817 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64dbf6-21af-4245-8e93-ef540b8701b5 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:54.336826 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e59134-5de5-4a4f-852d-e2eb2fee76a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:54.337656 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e59134-5de5-4a4f-852d-e2eb2fee76a3 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:54.338080 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e59134-5de5-4a4f-852d-e2eb2fee76a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:54.338524 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e59134-5de5-4a4f-852d-e2eb2fee76a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:54.339333 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e59134-5de5-4a4f-852d-e2eb2fee76a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:54.339804 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e59134-5de5-4a4f-852d-e2eb2fee76a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:54.341430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e59134-5de5-4a4f-852d-e2eb2fee76a3 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:54.353018 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 173/332] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 1 16:20:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:54.358452 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f4b35-865b-4723-bc3a-d129c3f9bac1 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:54.406151 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:54.484081 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f4b35-865b-4723-bc3a-d129c3f9bac1 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:54.528129 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f47283-09f1-4641-85a6-841976869c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:54.528796 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f47283-09f1-4641-85a6-841976869c2d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:54.529080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f47283-09f1-4641-85a6-841976869c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:54.529357 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f47283-09f1-4641-85a6-841976869c2d None admin] PATH_INFO: `/v3/users/730af05385b542238f3bac66d126bf87` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:54.533594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f47283-09f1-4641-85a6-841976869c2d None admin] RBAC: Authorizing `identity:get_user(user_id=730af05385b542238f3bac66d126bf87)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:54.535108 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f47283-09f1-4641-85a6-841976869c2d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:54.537044 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 160/333] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:54 2022] GET /identity/v3/users/730af05385b542238f3bac66d126bf87 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:54.542800 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e59134-5de5-4a4f-852d-e2eb2fee76a3 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:54.662559 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e59134-5de5-4a4f-852d-e2eb2fee76a3 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:54.681127 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:54.700608 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe738a2f-f3b2-4a4c-b521-1e9d76f01bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:54.701220 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe738a2f-f3b2-4a4c-b521-1e9d76f01bbc None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:54.701505 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe738a2f-f3b2-4a4c-b521-1e9d76f01bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:54.701777 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe738a2f-f3b2-4a4c-b521-1e9d76f01bbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:54.702537 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe738a2f-f3b2-4a4c-b521-1e9d76f01bbc None admin] RBAC: Adding query filter params (role.id=45545334e1fc473c9d85fc91a9b0a35c, user.id=730af05385b542238f3bac66d126bf87, scope.system=all) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:54.702849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe738a2f-f3b2-4a4c-b521-1e9d76f01bbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:54.704288 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe738a2f-f3b2-4a4c-b521-1e9d76f01bbc None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:54.716824 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 174/334] 10.222.0.18 () {56 vars in 1308 bytes} [Thu Sep 1 16:20:54 2022] GET /identity/v3/role_assignments?user.id=730af05385b542238f3bac66d126bf87&scope.system=all&role.id=45545334e1fc473c9d85fc91a9b0a35c => generated 510 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:55.467974 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f8b81-c19b-457a-a074-82219202eeb9 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:55.468947 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f8b81-c19b-457a-a074-82219202eeb9 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:55.469431 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f8b81-c19b-457a-a074-82219202eeb9 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:55.471244 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 161/335] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:55.476696 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff81331-ce3c-47de-8959-8f1251b89dfb None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:55.477250 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff81331-ce3c-47de-8959-8f1251b89dfb None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:55.477734 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff81331-ce3c-47de-8959-8f1251b89dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:55.496903 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-cff81331-ce3c-47de-8959-8f1251b89dfb None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:55.499213 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-cff81331-ce3c-47de-8959-8f1251b89dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:55.583108 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 175/336] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:56.118602 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f8b81-c19b-457a-a074-82219202eeb9 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:56.129102 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e6047-9a6e-461e-87e5-6d2cf51ec51e None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:56.129589 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e6047-9a6e-461e-87e5-6d2cf51ec51e None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:56.129662 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e6047-9a6e-461e-87e5-6d2cf51ec51e None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:56.131021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 176/337] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:56 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:56.138053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99295e6-869e-40dd-af24-870e81494017 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:56.138366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99295e6-869e-40dd-af24-870e81494017 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:56.138642 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99295e6-869e-40dd-af24-870e81494017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:56.180986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-c99295e6-869e-40dd-af24-870e81494017 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:56.182177 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-c99295e6-869e-40dd-af24-870e81494017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:56.330262 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 177/338] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:56.333471 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f8b81-c19b-457a-a074-82219202eeb9 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:56.342545 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:56.384383 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b12040e-5a43-466f-b4c0-4f7bd7b8da4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:56.385452 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b12040e-5a43-466f-b4c0-4f7bd7b8da4e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:56.385970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b12040e-5a43-466f-b4c0-4f7bd7b8da4e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:56.386473 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b12040e-5a43-466f-b4c0-4f7bd7b8da4e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:56.407190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b12040e-5a43-466f-b4c0-4f7bd7b8da4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:56.412555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-4b12040e-5a43-466f-b4c0-4f7bd7b8da4e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:20:56.412555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:56.412555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:56.412555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:56.412555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:56.412555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:56.412555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:56.412555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:56.412555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:56.412555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:56.412555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:56.412555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:56.412555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:56.412555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:56.412555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:56.412555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:56.413410 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:56.413410 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:56.413410 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:56.413410 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:56.413410 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:56.413410 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:56.413410 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:56.413410 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:56.413410 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:56.413410 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:56.413410 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:56.413410 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:56.413410 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:56.413410 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:56.413410 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:56.413410 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:56.414185 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:56.414185 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:56.414185 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:56.414185 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:56.414185 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:56.414185 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:56.414185 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:56.414185 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:56.414185 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:56.414185 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:56.414185 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:20:56.414185 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:20:56.417445 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 162/339] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:56.422779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99295e6-869e-40dd-af24-870e81494017 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:56.464391 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:56.549669 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99295e6-869e-40dd-af24-870e81494017 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:56.590120 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f1670-5932-4f9c-8559-37e4bc025d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:56.591663 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f1670-5932-4f9c-8559-37e4bc025d44 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:56.592154 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f1670-5932-4f9c-8559-37e4bc025d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:56.592622 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f1670-5932-4f9c-8559-37e4bc025d44 None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:56.593484 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f1670-5932-4f9c-8559-37e4bc025d44 None admin] RBAC: Adding query filter params (name=member) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:56.593983 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f1670-5932-4f9c-8559-37e4bc025d44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:56.595585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f1670-5932-4f9c-8559-37e4bc025d44 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:56.605859 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 178/340] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 1 16:20:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:56.611722 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b12040e-5a43-466f-b4c0-4f7bd7b8da4e None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:56.774178 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b12040e-5a43-466f-b4c0-4f7bd7b8da4e None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:56.781098 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93b008-e598-400c-b8f8-8d258cb0c4ea None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:56.781664 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93b008-e598-400c-b8f8-8d258cb0c4ea None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:56.782158 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93b008-e598-400c-b8f8-8d258cb0c4ea None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:56.784019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 179/341] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:20:56.790162 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd7623-645f-40e8-b301-67e9ea1b0137 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:56.790751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd7623-645f-40e8-b301-67e9ea1b0137 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:56.791131 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd7623-645f-40e8-b301-67e9ea1b0137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:56.825976 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba894bdd-b1c0-4fcd-a950-758eea6820a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:56.826857 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba894bdd-b1c0-4fcd-a950-758eea6820a0 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:56.827282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba894bdd-b1c0-4fcd-a950-758eea6820a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:56.828327 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-eedd7623-645f-40e8-b301-67e9ea1b0137 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:56.828775 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba894bdd-b1c0-4fcd-a950-758eea6820a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:56.831077 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-eedd7623-645f-40e8-b301-67e9ea1b0137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:56.887869 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:56.931450 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 180/342] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:20:56.933845 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd7623-645f-40e8-b301-67e9ea1b0137 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:56.956537 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba894bdd-b1c0-4fcd-a950-758eea6820a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:56.959385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba894bdd-b1c0-4fcd-a950-758eea6820a0 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:56.989591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:57.103620 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 163/343] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:20:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:20:57.110574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba894bdd-b1c0-4fcd-a950-758eea6820a0 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:57.145221 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd7623-645f-40e8-b301-67e9ea1b0137 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:57.201489 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:57.241979 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e3d03-0935-4f57-bd8d-edb0e28789d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:57.242684 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e3d03-0935-4f57-bd8d-edb0e28789d0 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:57.242965 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e3d03-0935-4f57-bd8d-edb0e28789d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:57.243238 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e3d03-0935-4f57-bd8d-edb0e28789d0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:57.243485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba894bdd-b1c0-4fcd-a950-758eea6820a0 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:57.279705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e3d03-0935-4f57-bd8d-edb0e28789d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:57.280803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011bc71-400f-4ee9-937c-7b517a31eec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:57.281505 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011bc71-400f-4ee9-937c-7b517a31eec5 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:57.281794 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011bc71-400f-4ee9-937c-7b517a31eec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:57.282066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011bc71-400f-4ee9-937c-7b517a31eec5 None admin] PATH_INFO: `/v3/projects/a3f472f7ed974397b7020b42b4f22bd3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:57.283241 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011bc71-400f-4ee9-937c-7b517a31eec5 None admin] RBAC: Authorizing `identity:get_project(project_id=a3f472f7ed974397b7020b42b4f22bd3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:57.284559 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-837e3d03-0935-4f57-bd8d-edb0e28789d0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:20:57.284559 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:57.284559 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:57.284559 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:57.284559 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:57.284559 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:57.284559 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:57.284559 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:57.284559 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:57.284559 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:57.284559 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:57.284559 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:57.284559 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:57.284559 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:57.284559 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:57.284559 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:57.285380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:57.285380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:57.285380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:57.285380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:57.285380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:57.285380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:57.285380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:57.285380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:57.285380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:57.285380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:57.285380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:57.285380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:57.285380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:57.285380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:57.285380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:57.285380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:57.286142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:57.286142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:57.286142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:57.286142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:57.286142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:57.286142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:57.286142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:57.286142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:57.286142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:57.286142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:57.286142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:20:57.286142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:20:57.286754 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011bc71-400f-4ee9-937c-7b517a31eec5 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:57.286820 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 181/344] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 1 16:20:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:57.286891 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 164/345] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:57 2022] GET /identity/v3/projects/a3f472f7ed974397b7020b42b4f22bd3 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:57.295637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e3d03-0935-4f57-bd8d-edb0e28789d0 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:57.295788 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011bc71-400f-4ee9-937c-7b517a31eec5 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:57.441794 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011bc71-400f-4ee9-937c-7b517a31eec5 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:57.463144 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:57.480059 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e3d03-0935-4f57-bd8d-edb0e28789d0 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:57.501611 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df1225-231c-4842-9872-6b2439225aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:57.502247 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df1225-231c-4842-9872-6b2439225aca None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:57.502532 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df1225-231c-4842-9872-6b2439225aca None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:57.502807 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df1225-231c-4842-9872-6b2439225aca None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:57.503589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df1225-231c-4842-9872-6b2439225aca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:57.503895 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df1225-231c-4842-9872-6b2439225aca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:57.505258 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df1225-231c-4842-9872-6b2439225aca None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:57.508056 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:57.518251 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b053d52-dedf-40f4-8a79-7c34699e45e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:57.518897 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b053d52-dedf-40f4-8a79-7c34699e45e2 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:57.519176 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b053d52-dedf-40f4-8a79-7c34699e45e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:57.519442 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b053d52-dedf-40f4-8a79-7c34699e45e2 None admin] PATH_INFO: `/v3/groups/c06650dbbcd04814bf81f9357cdc6ede` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:57.520206 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 165/346] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:20:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:57.521427 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cf754-e997-422f-bad9-471e95bfddb2 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:57.521710 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cf754-e997-422f-bad9-471e95bfddb2 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:57.521992 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cf754-e997-422f-bad9-471e95bfddb2 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:57.523464 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 166/347] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:57.524020 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4cf754-e997-422f-bad9-471e95bfddb2 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:57.525753 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b053d52-dedf-40f4-8a79-7c34699e45e2 None admin] RBAC: Authorizing `identity:get_group(group_id=c06650dbbcd04814bf81f9357cdc6ede)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:57.527238 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b053d52-dedf-40f4-8a79-7c34699e45e2 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:57.529689 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 182/348] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Sep 1 16:20:57 2022] GET /identity/v3/groups/c06650dbbcd04814bf81f9357cdc6ede => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:57.530487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b053d52-dedf-40f4-8a79-7c34699e45e2 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:57.656892 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b053d52-dedf-40f4-8a79-7c34699e45e2 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:57.665251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4cf754-e997-422f-bad9-471e95bfddb2 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:57.698055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c2b9b-c5fe-465a-9486-ae568fae4a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:57.698055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c2b9b-c5fe-465a-9486-ae568fae4a81 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:57.698410 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c2b9b-c5fe-465a-9486-ae568fae4a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:57.698735 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c2b9b-c5fe-465a-9486-ae568fae4a81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:57.714181 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a21e98-460c-4f6f-9433-ca96395e2b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:57.714859 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a21e98-460c-4f6f-9433-ca96395e2b92 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:57.715142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a21e98-460c-4f6f-9433-ca96395e2b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:57.715424 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a21e98-460c-4f6f-9433-ca96395e2b92 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:57.721083 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:57.732556 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a21e98-460c-4f6f-9433-ca96395e2b92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:57.738316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-37a21e98-460c-4f6f-9433-ca96395e2b92 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:20:57.738316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:57.738316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:57.738316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:57.738316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:57.738316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:57.738316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:57.738316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:57.738316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:57.738316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:57.738316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:57.738316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:57.738316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:57.738316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:57.738316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:57.738316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:57.739193 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:57.739193 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:57.739193 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:57.739193 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:57.739193 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:57.739193 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:57.739193 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:57.739193 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:57.739193 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:57.739193 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:57.739193 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:57.739193 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:57.739193 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:57.739193 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:57.739193 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:57.739193 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:57.739970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:57.739970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:57.739970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:57.739970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:57.739970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:57.739970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:57.739970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:57.739970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:57.739970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:57.739970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:57.739970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:20:57.739970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:20:57.743432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 167/349] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:57.744898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f3df2-97e5-4359-a8b7-3da206b10961 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:57.745195 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f3df2-97e5-4359-a8b7-3da206b10961 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:57.745477 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f3df2-97e5-4359-a8b7-3da206b10961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:57.767559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-867f3df2-97e5-4359-a8b7-3da206b10961 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:57.768779 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-867f3df2-97e5-4359-a8b7-3da206b10961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:57.789709 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c2b9b-c5fe-465a-9486-ae568fae4a81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:57.791407 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c2b9b-c5fe-465a-9486-ae568fae4a81 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:57.830172 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:57.849132 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 168/350] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:57.851387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f3df2-97e5-4359-a8b7-3da206b10961 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:57.915757 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 183/351] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:20:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:20:57.917090 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1230e7-0004-4095-9b13-b188872f7e7c None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:57.917388 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1230e7-0004-4095-9b13-b188872f7e7c None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:57.917673 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1230e7-0004-4095-9b13-b188872f7e7c None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:57.919068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 184/352] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:57.919835 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1230e7-0004-4095-9b13-b188872f7e7c None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:57.976446 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f3df2-97e5-4359-a8b7-3da206b10961 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:58.002624 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:58.030916 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07e91d-101b-4b96-b77b-07512cc02f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:58.031533 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07e91d-101b-4b96-b77b-07512cc02f9b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:58.031809 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07e91d-101b-4b96-b77b-07512cc02f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:58.032078 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07e91d-101b-4b96-b77b-07512cc02f9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:58.032808 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b07e91d-101b-4b96-b77b-07512cc02f9b None admin] RBAC: Adding query filter params (group.id=c06650dbbcd04814bf81f9357cdc6ede, role.id=45545334e1fc473c9d85fc91a9b0a35c, scope.project.id=a3f472f7ed974397b7020b42b4f22bd3) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:58.033113 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b07e91d-101b-4b96-b77b-07512cc02f9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:58.034570 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b07e91d-101b-4b96-b77b-07512cc02f9b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:58.040739 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 169/353] 10.222.0.18 () {56 vars in 1376 bytes} [Thu Sep 1 16:20:57 2022] GET /identity/v3/role_assignments?group.id=c06650dbbcd04814bf81f9357cdc6ede&scope.project.id=a3f472f7ed974397b7020b42b4f22bd3&role.id=45545334e1fc473c9d85fc91a9b0a35c => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:58.041700 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07e91d-101b-4b96-b77b-07512cc02f9b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:58.049721 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1230e7-0004-4095-9b13-b188872f7e7c None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:58.093691 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a77696-a1d3-4d3a-995b-c52d6cc94e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:58.094761 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a77696-a1d3-4d3a-995b-c52d6cc94e8d None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:58.095286 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a77696-a1d3-4d3a-995b-c52d6cc94e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:58.095787 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a77696-a1d3-4d3a-995b-c52d6cc94e8d None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:58.096759 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a77696-a1d3-4d3a-995b-c52d6cc94e8d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:58.097326 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a77696-a1d3-4d3a-995b-c52d6cc94e8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:58.099056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a77696-a1d3-4d3a-995b-c52d6cc94e8d None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:58.100775 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:58.111467 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 185/354] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 1 16:20:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:58.113338 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad94714-05bd-4f0a-868d-0bcc4938d433 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:58.113831 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad94714-05bd-4f0a-868d-0bcc4938d433 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:58.114336 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad94714-05bd-4f0a-868d-0bcc4938d433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:58.139830 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-0ad94714-05bd-4f0a-868d-0bcc4938d433 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:58.144288 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-0ad94714-05bd-4f0a-868d-0bcc4938d433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:58.207685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07e91d-101b-4b96-b77b-07512cc02f9b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:58.260332 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bd8cd-e128-435f-964c-9f81fd4d64c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:58.260976 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bd8cd-e128-435f-964c-9f81fd4d64c6 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:58.261257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bd8cd-e128-435f-964c-9f81fd4d64c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:58.261533 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bd8cd-e128-435f-964c-9f81fd4d64c6 None admin] PATH_INFO: `/v3/projects/a3f472f7ed974397b7020b42b4f22bd3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:58.263625 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5bd8cd-e128-435f-964c-9f81fd4d64c6 None admin] RBAC: Authorizing `identity:get_project(project_id=a3f472f7ed974397b7020b42b4f22bd3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:58.265082 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5bd8cd-e128-435f-964c-9f81fd4d64c6 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:58.267451 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 170/355] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:58 2022] GET /identity/v3/projects/a3f472f7ed974397b7020b42b4f22bd3 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:58.268241 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bd8cd-e128-435f-964c-9f81fd4d64c6 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:58.268670 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 186/356] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:58.270117 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad94714-05bd-4f0a-868d-0bcc4938d433 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:58.321608 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:58.380568 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad94714-05bd-4f0a-868d-0bcc4938d433 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:58.384043 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bd8cd-e128-435f-964c-9f81fd4d64c6 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:58.404616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:58.416167 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ea2e5-9842-44f5-ad88-95b79bbcbba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:58.416808 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ea2e5-9842-44f5-ad88-95b79bbcbba5 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:58.417082 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ea2e5-9842-44f5-ad88-95b79bbcbba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:58.417392 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ea2e5-9842-44f5-ad88-95b79bbcbba5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:58.421244 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c845d-8e38-4728-ab16-11c3abb65fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:58.422203 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c845d-8e38-4728-ab16-11c3abb65fd9 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:58.422686 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c845d-8e38-4728-ab16-11c3abb65fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:58.423137 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c845d-8e38-4728-ab16-11c3abb65fd9 None admin] PATH_INFO: `/v3/users/395e6a825d0d4402853bc35b9f656688` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:58.424466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c845d-8e38-4728-ab16-11c3abb65fd9 None admin] RBAC: Authorizing `identity:get_user(user_id=395e6a825d0d4402853bc35b9f656688)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:58.426313 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c845d-8e38-4728-ab16-11c3abb65fd9 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:58.428077 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 171/357] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:20:58 2022] GET /identity/v3/users/395e6a825d0d4402853bc35b9f656688 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:58.429061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c845d-8e38-4728-ab16-11c3abb65fd9 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:58.429481 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ea2e5-9842-44f5-ad88-95b79bbcbba5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:58.434526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-d56ea2e5-9842-44f5-ad88-95b79bbcbba5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:20:58.434526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:58.434526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:58.434526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:58.434526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:58.434526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:58.434526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:58.434526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:58.434526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:58.434526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:58.434526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:58.434526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:58.434526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:58.434526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:58.434526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:58.434526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:58.436037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:58.436037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:58.436037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:58.436037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:58.436037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:58.436037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:58.436037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:58.436037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:58.436037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:58.436037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:58.436037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:58.436037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:58.436037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:58.436037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:58.436037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:58.436037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:58.436788 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:58.436788 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:58.436788 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:58.436788 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:58.436788 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:58.436788 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:58.436788 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:58.436788 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:58.436788 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:58.436788 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:58.436788 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:20:58.436788 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:20:58.436788 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 187/358] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:58.437876 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc724a-4b67-4950-8423-b20312fbf4db None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:58.442342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc724a-4b67-4950-8423-b20312fbf4db None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:58.443178 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc724a-4b67-4950-8423-b20312fbf4db None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:58.444830 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 188/359] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:20:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:20:58.445222 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc724a-4b67-4950-8423-b20312fbf4db None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:58.543643 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c845d-8e38-4728-ab16-11c3abb65fd9 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:58.555072 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc724a-4b67-4950-8423-b20312fbf4db None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:58.556725 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:58.583645 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e84e90-222e-45fa-9b38-b6d1c740dccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:58.584736 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e84e90-222e-45fa-9b38-b6d1c740dccb None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:58.585231 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e84e90-222e-45fa-9b38-b6d1c740dccb None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:58.585703 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e84e90-222e-45fa-9b38-b6d1c740dccb None admin] PATH_INFO: `/v3/groups/c06650dbbcd04814bf81f9357cdc6ede` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:58.587258 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e84e90-222e-45fa-9b38-b6d1c740dccb None admin] RBAC: Authorizing `identity:get_group(group_id=c06650dbbcd04814bf81f9357cdc6ede)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:58.588903 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e84e90-222e-45fa-9b38-b6d1c740dccb None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:58.591548 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 172/360] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Sep 1 16:20:58 2022] GET /identity/v3/groups/c06650dbbcd04814bf81f9357cdc6ede => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:58.592527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04996097-b62f-4506-8cad-de772c88ff16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:58.592670 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e84e90-222e-45fa-9b38-b6d1c740dccb None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:58.593155 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04996097-b62f-4506-8cad-de772c88ff16 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:58.593419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04996097-b62f-4506-8cad-de772c88ff16 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:58.593692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04996097-b62f-4506-8cad-de772c88ff16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:58.597885 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04996097-b62f-4506-8cad-de772c88ff16 None admin] RBAC: Adding query filter params (role.id=12fe0aa7f1174edfb109315b88d6267b, user.id=395e6a825d0d4402853bc35b9f656688, scope.system=all) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:58.597885 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04996097-b62f-4506-8cad-de772c88ff16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:58.597885 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04996097-b62f-4506-8cad-de772c88ff16 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:58.610633 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 189/361] 10.222.0.18 () {56 vars in 1308 bytes} [Thu Sep 1 16:20:58 2022] GET /identity/v3/role_assignments?user.id=395e6a825d0d4402853bc35b9f656688&scope.system=all&role.id=12fe0aa7f1174edfb109315b88d6267b => generated 510 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:58.612034 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d4ef5-da76-44b0-a10e-9a368d47a575 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:58.612271 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d4ef5-da76-44b0-a10e-9a368d47a575 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:58.612551 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d4ef5-da76-44b0-a10e-9a368d47a575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:58.632912 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-941d4ef5-da76-44b0-a10e-9a368d47a575 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:20:58.634121 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-941d4ef5-da76-44b0-a10e-9a368d47a575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:20:58.665716 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:58.717108 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 190/362] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:20:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:20:58.717946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d4ef5-da76-44b0-a10e-9a368d47a575 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:58.723141 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e84e90-222e-45fa-9b38-b6d1c740dccb None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:58.762863 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a4a6e1-f309-4998-b49b-5da0647c8920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:58.763987 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4a6e1-f309-4998-b49b-5da0647c8920 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:58.764468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4a6e1-f309-4998-b49b-5da0647c8920 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:58.764938 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4a6e1-f309-4998-b49b-5da0647c8920 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:58.765920 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a4a6e1-f309-4998-b49b-5da0647c8920 None admin] RBAC: Adding query filter params (name=member) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:58.766480 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a4a6e1-f309-4998-b49b-5da0647c8920 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:58.768215 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a4a6e1-f309-4998-b49b-5da0647c8920 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:58.779151 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 173/363] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 1 16:20:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:58.780527 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a4a6e1-f309-4998-b49b-5da0647c8920 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:58.823470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:58.836546 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d4ef5-da76-44b0-a10e-9a368d47a575 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:58.876717 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf35d18-dabd-429d-95ca-7b2b646b7c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:58.877383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf35d18-dabd-429d-95ca-7b2b646b7c30 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:58.877624 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf35d18-dabd-429d-95ca-7b2b646b7c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:58.877895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf35d18-dabd-429d-95ca-7b2b646b7c30 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:58.890596 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf35d18-dabd-429d-95ca-7b2b646b7c30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:58.895854 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-ddf35d18-dabd-429d-95ca-7b2b646b7c30 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:20:58.895854 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:58.895854 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:58.895854 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:58.895854 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:58.895854 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:58.895854 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:58.895854 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:58.895854 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:58.895854 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:58.895854 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:58.895854 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:58.895854 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:58.895854 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:58.895854 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:58.895854 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:58.896733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:58.896733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:58.896733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:58.896733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:58.896733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:58.896733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:58.896733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:58.896733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:58.896733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:58.896733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:58.896733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:58.896733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:58.896733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:58.896733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:58.896733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:58.896733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:58.897513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:58.897513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:58.897513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:58.897513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:58.897513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:58.897513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:58.897513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:58.897513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:58.897513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:58.897513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:58.897513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:20:58.897513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:20:58.898099 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 191/364] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 1 16:20:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:58.898791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf35d18-dabd-429d-95ca-7b2b646b7c30 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:58.900243 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a4a6e1-f309-4998-b49b-5da0647c8920 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:58.920549 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:58.942427 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6049c-0443-4b27-b189-b746223f6b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:58.943389 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6049c-0443-4b27-b189-b746223f6b67 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:58.943872 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6049c-0443-4b27-b189-b746223f6b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:58.944344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6049c-0443-4b27-b189-b746223f6b67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:58.945296 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6049c-0443-4b27-b189-b746223f6b67 None admin] RBAC: Adding query filter params (group.id=c06650dbbcd04814bf81f9357cdc6ede, role.id=f2a6e936aa294323ba7087bf377276b2, scope.project.id=a3f472f7ed974397b7020b42b4f22bd3) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:58.945805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6049c-0443-4b27-b189-b746223f6b67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:58.947490 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6049c-0443-4b27-b189-b746223f6b67 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:58.953694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 174/365] 10.222.0.18 () {56 vars in 1376 bytes} [Thu Sep 1 16:20:58 2022] GET /identity/v3/role_assignments?group.id=c06650dbbcd04814bf81f9357cdc6ede&scope.project.id=a3f472f7ed974397b7020b42b4f22bd3&role.id=f2a6e936aa294323ba7087bf377276b2 => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:58.954796 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6049c-0443-4b27-b189-b746223f6b67 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:59.019511 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf35d18-dabd-429d-95ca-7b2b646b7c30 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:59.060847 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2d5a2-86d4-40b7-93fd-4f1cb919db65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:59.061507 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2d5a2-86d4-40b7-93fd-4f1cb919db65 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:59.061793 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2d5a2-86d4-40b7-93fd-4f1cb919db65 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:59.062065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2d5a2-86d4-40b7-93fd-4f1cb919db65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:59.075140 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6049c-0443-4b27-b189-b746223f6b67 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:59.098743 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:59.117949 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e029d96-21ee-4b7a-bf72-cbd418485f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:59.118605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e029d96-21ee-4b7a-bf72-cbd418485f21 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:59.118891 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e029d96-21ee-4b7a-bf72-cbd418485f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:59.119164 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e029d96-21ee-4b7a-bf72-cbd418485f21 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:59.119846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e029d96-21ee-4b7a-bf72-cbd418485f21 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:59.120154 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e029d96-21ee-4b7a-bf72-cbd418485f21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:59.121625 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e029d96-21ee-4b7a-bf72-cbd418485f21 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:59.132304 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 175/366] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:20:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:59.137481 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e029d96-21ee-4b7a-bf72-cbd418485f21 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:59.143974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c2d5a2-86d4-40b7-93fd-4f1cb919db65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:59.145583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c2d5a2-86d4-40b7-93fd-4f1cb919db65 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:59.202135 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:59.274205 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 192/367] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:20:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:20:59.275087 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2d5a2-86d4-40b7-93fd-4f1cb919db65 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:59.290330 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e029d96-21ee-4b7a-bf72-cbd418485f21 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:59.337171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87541d-8494-4488-8af0-84f967e8a62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:59.337722 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87541d-8494-4488-8af0-84f967e8a62b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:59.338002 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87541d-8494-4488-8af0-84f967e8a62b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:59.339291 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87541d-8494-4488-8af0-84f967e8a62b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:59.346378 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:59.405442 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2d5a2-86d4-40b7-93fd-4f1cb919db65 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:59.421142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f87541d-8494-4488-8af0-84f967e8a62b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:59.422859 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f87541d-8494-4488-8af0-84f967e8a62b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:59.461453 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e726ffc6-e3cd-4d13-98db-e4ce328fea57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:59.462492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726ffc6-e3cd-4d13-98db-e4ce328fea57 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:59.462999 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726ffc6-e3cd-4d13-98db-e4ce328fea57 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:59.463475 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726ffc6-e3cd-4d13-98db-e4ce328fea57 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:59.477754 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:59.482764 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e726ffc6-e3cd-4d13-98db-e4ce328fea57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:20:59.491516 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-e726ffc6-e3cd-4d13-98db-e4ce328fea57 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:20:59.491516 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:20:59.491516 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:20:59.491516 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:20:59.491516 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:20:59.491516 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:20:59.491516 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:20:59.491516 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:20:59.491516 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:20:59.491516 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:20:59.491516 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:20:59.491516 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:20:59.491516 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:20:59.491516 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:20:59.491516 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:20:59.491516 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:20:59.492432 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:20:59.492432 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:20:59.492432 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:20:59.492432 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:20:59.492432 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:20:59.492432 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:20:59.492432 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:20:59.492432 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:20:59.492432 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:20:59.492432 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:20:59.492432 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:20:59.492432 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:20:59.492432 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:20:59.492432 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:20:59.492432 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:20:59.492432 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:20:59.493225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:20:59.493225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:20:59.493225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:20:59.493225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:20:59.493225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:20:59.493225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:20:59.493225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:20:59.493225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:20:59.493225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:20:59.493225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:20:59.493225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:20:59.493225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:20:59.499680 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 193/368] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:20:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:20:59.499680 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e726ffc6-e3cd-4d13-98db-e4ce328fea57 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:59.554172 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 176/369] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:20:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 16:20:59.555431 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87541d-8494-4488-8af0-84f967e8a62b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:59.627636 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e726ffc6-e3cd-4d13-98db-e4ce328fea57 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:59.642432 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:59.670624 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567623a0-cc9f-4a33-90de-7cd987b4e1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:59.670975 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87541d-8494-4488-8af0-84f967e8a62b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:59.671393 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567623a0-cc9f-4a33-90de-7cd987b4e1e2 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:59.671530 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567623a0-cc9f-4a33-90de-7cd987b4e1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:59.671801 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567623a0-cc9f-4a33-90de-7cd987b4e1e2 None admin] PATH_INFO: `/v3/projects/cc5b99bf5631484eb25353883854e856` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:59.673646 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567623a0-cc9f-4a33-90de-7cd987b4e1e2 None admin] RBAC: Authorizing `identity:get_project(project_id=cc5b99bf5631484eb25353883854e856)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:59.675239 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567623a0-cc9f-4a33-90de-7cd987b4e1e2 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:59.677197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 194/370] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:59 2022] GET /identity/v3/projects/cc5b99bf5631484eb25353883854e856 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:59.677986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567623a0-cc9f-4a33-90de-7cd987b4e1e2 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:59.710202 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a5baa-d06e-4498-9453-332adb5a93af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:59.711255 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a5baa-d06e-4498-9453-332adb5a93af None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:59.711807 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a5baa-d06e-4498-9453-332adb5a93af None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:59.712258 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a5baa-d06e-4498-9453-332adb5a93af None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:59.713119 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a5baa-d06e-4498-9453-332adb5a93af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:20:59.713703 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a5baa-d06e-4498-9453-332adb5a93af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:59.715388 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a5baa-d06e-4498-9453-332adb5a93af None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:59.724602 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 177/371] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:20:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:59.725871 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a5baa-d06e-4498-9453-332adb5a93af None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:59.730939 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:59.795715 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567623a0-cc9f-4a33-90de-7cd987b4e1e2 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:59.836700 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402999c5-a5ee-49ba-b829-c8d13c734367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:59.837116 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a5baa-d06e-4498-9453-332adb5a93af None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:20:59.837962 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402999c5-a5ee-49ba-b829-c8d13c734367 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:59.838558 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402999c5-a5ee-49ba-b829-c8d13c734367 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:59.839070 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402999c5-a5ee-49ba-b829-c8d13c734367 None admin] PATH_INFO: `/v3/projects/cc5b99bf5631484eb25353883854e856` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:59.840825 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402999c5-a5ee-49ba-b829-c8d13c734367 None admin] RBAC: Authorizing `identity:get_project(project_id=cc5b99bf5631484eb25353883854e856)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:59.842806 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402999c5-a5ee-49ba-b829-c8d13c734367 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:59.849860 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 195/372] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:20:59 2022] GET /identity/v3/projects/cc5b99bf5631484eb25353883854e856 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:59.851463 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402999c5-a5ee-49ba-b829-c8d13c734367 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:59.877840 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e9b0c-25d7-467d-b2f1-8805156fcf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:20:59.878454 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e9b0c-25d7-467d-b2f1-8805156fcf1b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:20:59.878748 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e9b0c-25d7-467d-b2f1-8805156fcf1b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:20:59.879033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e9b0c-25d7-467d-b2f1-8805156fcf1b None admin] PATH_INFO: `/v3/groups/c06650dbbcd04814bf81f9357cdc6ede` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:20:59.880278 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e9b0c-25d7-467d-b2f1-8805156fcf1b None admin] RBAC: Authorizing `identity:get_group(group_id=c06650dbbcd04814bf81f9357cdc6ede)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:20:59.881966 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e9b0c-25d7-467d-b2f1-8805156fcf1b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:20:59.883603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 178/373] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Sep 1 16:20:59 2022] GET /identity/v3/groups/c06650dbbcd04814bf81f9357cdc6ede => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:20:59.884597 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e9b0c-25d7-467d-b2f1-8805156fcf1b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:20:59.914535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:59.974603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:20:59.983264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402999c5-a5ee-49ba-b829-c8d13c734367 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:00.011429 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e9b0c-25d7-467d-b2f1-8805156fcf1b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:00.035290 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b849a3-f348-4849-90bb-8d4e1fbad183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:00.035914 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b849a3-f348-4849-90bb-8d4e1fbad183 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:00.036205 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b849a3-f348-4849-90bb-8d4e1fbad183 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:00.036484 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b849a3-f348-4849-90bb-8d4e1fbad183 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:00.054291 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a7297-09ef-4e9e-b689-65fae0b1003b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:00.055182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a7297-09ef-4e9e-b689-65fae0b1003b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:00.055540 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a7297-09ef-4e9e-b689-65fae0b1003b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:00.055984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a7297-09ef-4e9e-b689-65fae0b1003b None admin] PATH_INFO: `/v3/groups/c06650dbbcd04814bf81f9357cdc6ede` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:00.057188 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29a7297-09ef-4e9e-b689-65fae0b1003b None admin] RBAC: Authorizing `identity:get_group(group_id=c06650dbbcd04814bf81f9357cdc6ede)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:00.060668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29a7297-09ef-4e9e-b689-65fae0b1003b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:00.062960 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 179/374] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Sep 1 16:20:59 2022] GET /identity/v3/groups/c06650dbbcd04814bf81f9357cdc6ede => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:00.063905 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a7297-09ef-4e9e-b689-65fae0b1003b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:00.128072 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b849a3-f348-4849-90bb-8d4e1fbad183 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:00.129916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b849a3-f348-4849-90bb-8d4e1fbad183 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:00.191945 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a7297-09ef-4e9e-b689-65fae0b1003b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:00.197969 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:00.233294 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c0f67-5464-4d4a-9102-53292a9833b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:00.233919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c0f67-5464-4d4a-9102-53292a9833b8 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:00.234202 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c0f67-5464-4d4a-9102-53292a9833b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:00.234504 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c0f67-5464-4d4a-9102-53292a9833b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:00.235242 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51c0f67-5464-4d4a-9102-53292a9833b8 None admin] RBAC: Adding query filter params (group.id=c06650dbbcd04814bf81f9357cdc6ede, role.id=45545334e1fc473c9d85fc91a9b0a35c, scope.project.id=cc5b99bf5631484eb25353883854e856) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:00.235550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51c0f67-5464-4d4a-9102-53292a9833b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:00.237450 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51c0f67-5464-4d4a-9102-53292a9833b8 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:00.245507 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 180/375] 10.222.0.18 () {56 vars in 1376 bytes} [Thu Sep 1 16:21:00 2022] GET /identity/v3/role_assignments?group.id=c06650dbbcd04814bf81f9357cdc6ede&scope.project.id=cc5b99bf5631484eb25353883854e856&role.id=45545334e1fc473c9d85fc91a9b0a35c => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:00.246170 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c0f67-5464-4d4a-9102-53292a9833b8 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:00.251590 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:00.269147 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 196/376] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:20:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:21:00.283153 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b849a3-f348-4849-90bb-8d4e1fbad183 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:00.411941 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b849a3-f348-4849-90bb-8d4e1fbad183 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:00.457421 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16c758-6eb0-4a23-9d3b-9f7295e39daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:00.459007 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16c758-6eb0-4a23-9d3b-9f7295e39daa None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:00.459493 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16c758-6eb0-4a23-9d3b-9f7295e39daa None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:00.459960 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16c758-6eb0-4a23-9d3b-9f7295e39daa None admin] PATH_INFO: `/v3/projects/0d95c560fa484406b7dcc65daad0e99e` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:00.461582 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a16c758-6eb0-4a23-9d3b-9f7295e39daa None admin] RBAC: Authorizing `identity:get_project(project_id=0d95c560fa484406b7dcc65daad0e99e)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:00.463505 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a16c758-6eb0-4a23-9d3b-9f7295e39daa None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:00.467277 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 197/377] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:21:00 2022] GET /identity/v3/projects/0d95c560fa484406b7dcc65daad0e99e => generated 344 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:00.472905 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16c758-6eb0-4a23-9d3b-9f7295e39daa None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:00.476867 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c0f67-5464-4d4a-9102-53292a9833b8 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:00.493687 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:00.513600 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26cecd4-1181-4639-b354-5b50f80bf851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:00.514218 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cecd4-1181-4639-b354-5b50f80bf851 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:00.514611 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cecd4-1181-4639-b354-5b50f80bf851 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:00.514887 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cecd4-1181-4639-b354-5b50f80bf851 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:00.515673 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26cecd4-1181-4639-b354-5b50f80bf851 None admin] RBAC: Adding query filter params (group.id=c06650dbbcd04814bf81f9357cdc6ede, role.id=f2a6e936aa294323ba7087bf377276b2, scope.project.id=cc5b99bf5631484eb25353883854e856) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:00.515981 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26cecd4-1181-4639-b354-5b50f80bf851 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:00.517471 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26cecd4-1181-4639-b354-5b50f80bf851 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:00.523743 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 181/378] 10.222.0.18 () {56 vars in 1376 bytes} [Thu Sep 1 16:21:00 2022] GET /identity/v3/role_assignments?group.id=c06650dbbcd04814bf81f9357cdc6ede&scope.project.id=cc5b99bf5631484eb25353883854e856&role.id=f2a6e936aa294323ba7087bf377276b2 => generated 260 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:00.527863 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69054f-943a-4ce2-a708-304b66e8ca15 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:00.527863 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69054f-943a-4ce2-a708-304b66e8ca15 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:00.527863 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69054f-943a-4ce2-a708-304b66e8ca15 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:00.527863 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 182/379] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:00.533569 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1390b-f587-4869-8207-6dc81973790c None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:00.533877 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1390b-f587-4869-8207-6dc81973790c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:00.534117 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1390b-f587-4869-8207-6dc81973790c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:00.569287 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-47f1390b-f587-4869-8207-6dc81973790c None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:00.571365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-47f1390b-f587-4869-8207-6dc81973790c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:00.650275 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16c758-6eb0-4a23-9d3b-9f7295e39daa None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:00.707414 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:00.717524 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeca29d-cce4-48a0-8dd8-cc92bea6448b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:00.718180 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeca29d-cce4-48a0-8dd8-cc92bea6448b None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:00.718502 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeca29d-cce4-48a0-8dd8-cc92bea6448b None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:00.718791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeca29d-cce4-48a0-8dd8-cc92bea6448b None admin] PATH_INFO: `/v3/groups/c486df90c83c4eca9b47e8c3dc2bd0f6` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:00.720824 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 183/380] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:21:00.727291 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeca29d-cce4-48a0-8dd8-cc92bea6448b None admin] RBAC: Authorizing `identity:get_group(group_id=c486df90c83c4eca9b47e8c3dc2bd0f6)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:00.728766 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeca29d-cce4-48a0-8dd8-cc92bea6448b None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:00.730731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 198/381] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Sep 1 16:21:00 2022] GET /identity/v3/groups/c486df90c83c4eca9b47e8c3dc2bd0f6 => generated 229 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:00.736242 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1390b-f587-4869-8207-6dc81973790c None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:00.850753 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1390b-f587-4869-8207-6dc81973790c None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:00.892634 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee06478-d2d4-40ef-bd24-c129f17e248b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:00.893254 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee06478-d2d4-40ef-bd24-c129f17e248b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:00.893533 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee06478-d2d4-40ef-bd24-c129f17e248b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:00.893805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee06478-d2d4-40ef-bd24-c129f17e248b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:00.894628 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee06478-d2d4-40ef-bd24-c129f17e248b None admin] RBAC: Adding query filter params (group.id=c486df90c83c4eca9b47e8c3dc2bd0f6, role.id=e8ba063905484d22993bcf04830091a3, scope.project.id=0d95c560fa484406b7dcc65daad0e99e) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:00.894945 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee06478-d2d4-40ef-bd24-c129f17e248b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:00.896372 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee06478-d2d4-40ef-bd24-c129f17e248b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:00.902325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 184/382] 10.222.0.18 () {56 vars in 1376 bytes} [Thu Sep 1 16:21:00 2022] GET /identity/v3/role_assignments?group.id=c486df90c83c4eca9b47e8c3dc2bd0f6&scope.project.id=0d95c560fa484406b7dcc65daad0e99e&role.id=e8ba063905484d22993bcf04830091a3 => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:21:01.117983 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeca29d-cce4-48a0-8dd8-cc92bea6448b None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:01.211336 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:01.234911 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeca29d-cce4-48a0-8dd8-cc92bea6448b None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:01.273007 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff21274f-1ad2-469f-88ba-bebb0dae480d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:01.273648 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21274f-1ad2-469f-88ba-bebb0dae480d None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:01.273932 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21274f-1ad2-469f-88ba-bebb0dae480d None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:01.274210 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21274f-1ad2-469f-88ba-bebb0dae480d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:01.286597 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff21274f-1ad2-469f-88ba-bebb0dae480d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:01.292014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-ff21274f-1ad2-469f-88ba-bebb0dae480d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:21:01.292014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:01.292014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:01.292014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:01.292014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:01.292014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:01.292014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:01.292014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:01.292014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:01.292014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:01.292014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:01.292014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:01.292014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:01.292014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:01.292014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:01.292014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:01.292849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:01.292849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:01.292849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:01.292849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:01.292849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:01.292849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:01.292849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:01.292849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:01.292849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:01.292849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:01.292849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:01.292849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:01.292849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:01.292849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:01.292849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:01.292849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:01.294628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:01.294628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:01.294628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:01.294628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:01.294628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:01.294628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:01.294628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:01.294628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:01.294628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:01.294628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:01.294628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:21:01.294628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:01.294628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 199/383] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:21:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:01.299680 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee06478-d2d4-40ef-bd24-c129f17e248b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:01.390437 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf897e-5027-4195-8c93-98f894679668 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:01.390713 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf897e-5027-4195-8c93-98f894679668 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:01.390903 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf897e-5027-4195-8c93-98f894679668 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:01.392397 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 200/384] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:01.397221 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79e98d-8538-4894-908f-6d368789c43d None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:01.397498 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:01.397983 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79e98d-8538-4894-908f-6d368789c43d None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:01.398544 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79e98d-8538-4894-908f-6d368789c43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:01.436772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-cd79e98d-8538-4894-908f-6d368789c43d None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:01.438019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-cd79e98d-8538-4894-908f-6d368789c43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:01.444569 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee06478-d2d4-40ef-bd24-c129f17e248b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:01.488943 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23430336-11f2-4e03-b491-ed324c0c81eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:01.489569 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23430336-11f2-4e03-b491-ed324c0c81eb None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:01.489852 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23430336-11f2-4e03-b491-ed324c0c81eb None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:01.490127 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23430336-11f2-4e03-b491-ed324c0c81eb None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:01.490824 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23430336-11f2-4e03-b491-ed324c0c81eb None admin] RBAC: Adding query filter params (name=member) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:01.491130 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23430336-11f2-4e03-b491-ed324c0c81eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:01.492497 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23430336-11f2-4e03-b491-ed324c0c81eb None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:01.512036 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 185/385] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 1 16:21:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:01.516224 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23430336-11f2-4e03-b491-ed324c0c81eb None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:01.581474 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 201/386] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:21:01.688109 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23430336-11f2-4e03-b491-ed324c0c81eb None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:01.723237 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:01.735649 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768660b-ba59-4b50-8ddf-05d534d7fbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:01.736290 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768660b-ba59-4b50-8ddf-05d534d7fbb4 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:01.736569 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768660b-ba59-4b50-8ddf-05d534d7fbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:01.736849 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768660b-ba59-4b50-8ddf-05d534d7fbb4 None admin] PATH_INFO: `/v3/groups/c06650dbbcd04814bf81f9357cdc6ede` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:01.737976 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f768660b-ba59-4b50-8ddf-05d534d7fbb4 None admin] RBAC: Authorizing `identity:get_group(group_id=c06650dbbcd04814bf81f9357cdc6ede)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:01.739423 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f768660b-ba59-4b50-8ddf-05d534d7fbb4 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:01.741343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 186/387] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Sep 1 16:21:01 2022] GET /identity/v3/groups/c06650dbbcd04814bf81f9357cdc6ede => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:01.746591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd79e98d-8538-4894-908f-6d368789c43d None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:01.842195 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:01.883292 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd79e98d-8538-4894-908f-6d368789c43d None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:01.927319 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23140788-571e-4150-bd6f-fd0b3416d6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:01.928304 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23140788-571e-4150-bd6f-fd0b3416d6df None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:01.928788 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23140788-571e-4150-bd6f-fd0b3416d6df None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:01.929302 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23140788-571e-4150-bd6f-fd0b3416d6df None admin] PATH_INFO: `/v3/projects/a3f472f7ed974397b7020b42b4f22bd3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:01.930660 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23140788-571e-4150-bd6f-fd0b3416d6df None admin] RBAC: Authorizing `identity:get_project(project_id=a3f472f7ed974397b7020b42b4f22bd3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:01.932322 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23140788-571e-4150-bd6f-fd0b3416d6df None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:01.935293 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 202/388] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:21:01 2022] GET /identity/v3/projects/a3f472f7ed974397b7020b42b4f22bd3 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:01.942331 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768660b-ba59-4b50-8ddf-05d534d7fbb4 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:01.960443 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23140788-571e-4150-bd6f-fd0b3416d6df None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:02.065127 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768660b-ba59-4b50-8ddf-05d534d7fbb4 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:02.111000 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f459742-e56c-4f60-903c-a6eee0e9a44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:02.111859 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f459742-e56c-4f60-903c-a6eee0e9a44f None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:02.112209 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f459742-e56c-4f60-903c-a6eee0e9a44f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:02.112542 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f459742-e56c-4f60-903c-a6eee0e9a44f None admin] PATH_INFO: `/v3/projects/a3f472f7ed974397b7020b42b4f22bd3/groups/c06650dbbcd04814bf81f9357cdc6ede/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:02.114483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f459742-e56c-4f60-903c-a6eee0e9a44f None admin] RBAC: Authorizing `identity:create_grant(project_id=a3f472f7ed974397b7020b42b4f22bd3, group_id=c06650dbbcd04814bf81f9357cdc6ede, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:02.116165 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f459742-e56c-4f60-903c-a6eee0e9a44f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:02.145453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 187/389] 10.222.0.18 () {58 vars in 1442 bytes} [Thu Sep 1 16:21:01 2022] PUT /identity/v3/projects/a3f472f7ed974397b7020b42b4f22bd3/groups/c06650dbbcd04814bf81f9357cdc6ede/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:21:02.170399 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23140788-571e-4150-bd6f-fd0b3416d6df None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:02.210255 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd80b7b5-d769-42bf-bf45-ac19e77b1ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:02.210845 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80b7b5-d769-42bf-bf45-ac19e77b1ff1 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:02.211126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80b7b5-d769-42bf-bf45-ac19e77b1ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:02.211398 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80b7b5-d769-42bf-bf45-ac19e77b1ff1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:02.214633 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:02.225693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd80b7b5-d769-42bf-bf45-ac19e77b1ff1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:02.231255 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-fd80b7b5-d769-42bf-bf45-ac19e77b1ff1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:21:02.231255 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:02.231255 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:02.231255 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:02.231255 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:02.231255 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:02.231255 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:02.231255 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:02.231255 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:02.231255 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:02.231255 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:02.231255 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:02.231255 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:02.231255 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:02.231255 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:02.231255 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:02.232094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:02.232094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:02.232094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:02.232094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:02.232094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:02.232094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:02.232094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:02.232094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:02.232094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:02.232094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:02.232094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:02.232094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:02.232094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:02.232094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:02.232094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:02.232094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:02.233281 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:02.233281 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:02.233281 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:02.233281 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:02.233281 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:02.233281 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:02.233281 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:02.233281 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:02.233281 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:02.233281 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:02.233281 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:21:02.233281 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:02.233281 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 203/390] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 1 16:21:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:02.238148 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f459742-e56c-4f60-903c-a6eee0e9a44f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:02.300380 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:02.368565 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f459742-e56c-4f60-903c-a6eee0e9a44f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:02.409377 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7879746f-f1ad-4591-92bd-f51849f73278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:02.410009 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879746f-f1ad-4591-92bd-f51849f73278 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:02.410327 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879746f-f1ad-4591-92bd-f51849f73278 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:02.410606 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879746f-f1ad-4591-92bd-f51849f73278 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:02.411278 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7879746f-f1ad-4591-92bd-f51849f73278 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:02.411584 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7879746f-f1ad-4591-92bd-f51849f73278 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:02.412974 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7879746f-f1ad-4591-92bd-f51849f73278 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:02.422408 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 188/391] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:02.431519 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd80b7b5-d769-42bf-bf45-ac19e77b1ff1 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:02.545731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd80b7b5-d769-42bf-bf45-ac19e77b1ff1 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:02.582747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7efe9-660e-4154-8dff-312653872ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:02.583384 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7efe9-660e-4154-8dff-312653872ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:02.583662 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7efe9-660e-4154-8dff-312653872ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:02.583935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7efe9-660e-4154-8dff-312653872ed5 None admin] PATH_INFO: `/v3/groups/c06650dbbcd04814bf81f9357cdc6ede` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:02.585044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea7efe9-660e-4154-8dff-312653872ed5 None admin] RBAC: Authorizing `identity:get_group(group_id=c06650dbbcd04814bf81f9357cdc6ede)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:02.586591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea7efe9-660e-4154-8dff-312653872ed5 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:02.588580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 204/392] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Sep 1 16:21:02 2022] GET /identity/v3/groups/c06650dbbcd04814bf81f9357cdc6ede => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:02.593701 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7879746f-f1ad-4591-92bd-f51849f73278 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:02.709869 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7879746f-f1ad-4591-92bd-f51849f73278 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:02.729230 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:02.753381 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9531df0-6ae4-410b-98fe-de494f0160a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:02.754021 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9531df0-6ae4-410b-98fe-de494f0160a1 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:02.754344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9531df0-6ae4-410b-98fe-de494f0160a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:02.754619 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9531df0-6ae4-410b-98fe-de494f0160a1 None admin] PATH_INFO: `/v3/projects/a3f472f7ed974397b7020b42b4f22bd3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:02.758955 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9531df0-6ae4-410b-98fe-de494f0160a1 None admin] RBAC: Authorizing `identity:get_project(project_id=a3f472f7ed974397b7020b42b4f22bd3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:02.760408 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9531df0-6ae4-410b-98fe-de494f0160a1 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:02.762460 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 189/393] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:21:02 2022] GET /identity/v3/projects/a3f472f7ed974397b7020b42b4f22bd3 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:21:02.767834 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7efe9-660e-4154-8dff-312653872ed5 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:02.885580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:02.930009 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7efe9-660e-4154-8dff-312653872ed5 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:02.966506 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec6e2a-4658-424c-835c-19004d108c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:02.967180 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec6e2a-4658-424c-835c-19004d108c88 None admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:02.967461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec6e2a-4658-424c-835c-19004d108c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:02.967734 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec6e2a-4658-424c-835c-19004d108c88 None admin] PATH_INFO: `/v3/projects/a3f472f7ed974397b7020b42b4f22bd3/groups/c06650dbbcd04814bf81f9357cdc6ede/roles/f2a6e936aa294323ba7087bf377276b2` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:02.971831 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec6e2a-4658-424c-835c-19004d108c88 None admin] RBAC: Authorizing `identity:create_grant(project_id=a3f472f7ed974397b7020b42b4f22bd3, group_id=c06650dbbcd04814bf81f9357cdc6ede, role_id=f2a6e936aa294323ba7087bf377276b2)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:02.972190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0de162-da12-4e7a-9901-e8c598a904cf None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:02.972481 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0de162-da12-4e7a-9901-e8c598a904cf None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:02.972760 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0de162-da12-4e7a-9901-e8c598a904cf None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:02.973339 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec6e2a-4658-424c-835c-19004d108c88 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:02.974260 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 190/394] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:02.983565 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c38f56-a3b3-434b-9b6f-b22738f1f8c1 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:02.983848 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c38f56-a3b3-434b-9b6f-b22738f1f8c1 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:02.984126 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c38f56-a3b3-434b-9b6f-b22738f1f8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:03.001145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 205/395] 10.222.0.18 () {58 vars in 1442 bytes} [Thu Sep 1 16:21:02 2022] PUT /identity/v3/projects/a3f472f7ed974397b7020b42b4f22bd3/groups/c06650dbbcd04814bf81f9357cdc6ede/roles/f2a6e936aa294323ba7087bf377276b2 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:21:03.023399 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-87c38f56-a3b3-434b-9b6f-b22738f1f8c1 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:03.024636 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-87c38f56-a3b3-434b-9b6f-b22738f1f8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:03.121257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 191/396] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:21:03.300525 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b1373-7ed2-46d2-b2df-6a64c4a5231d None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:03.300897 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b1373-7ed2-46d2-b2df-6a64c4a5231d None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:03.301131 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b1373-7ed2-46d2-b2df-6a64c4a5231d None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:03.303467 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 206/397] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:03.307177 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eec036-b1dd-485b-b1d3-d431c6dabff0 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:03.307466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eec036-b1dd-485b-b1d3-d431c6dabff0 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:03.307805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eec036-b1dd-485b-b1d3-d431c6dabff0 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:03.309377 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 192/398] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:21:03.311200 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb30ee9-5ffb-49e4-8997-8a23623fd70a None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:03.311507 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb30ee9-5ffb-49e4-8997-8a23623fd70a None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:03.311905 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb30ee9-5ffb-49e4-8997-8a23623fd70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:03.316346 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b160b13c-f3ac-472a-9e84-16ad6dbe6ce1 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:03.316638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b160b13c-f3ac-472a-9e84-16ad6dbe6ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:03.316913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b160b13c-f3ac-472a-9e84-16ad6dbe6ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:03.341771 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-b160b13c-f3ac-472a-9e84-16ad6dbe6ce1 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:03.343000 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b160b13c-f3ac-472a-9e84-16ad6dbe6ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:03.353024 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-2cb30ee9-5ffb-49e4-8997-8a23623fd70a None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:03.354318 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-2cb30ee9-5ffb-49e4-8997-8a23623fd70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:03.454860 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 193/399] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:21:03.472670 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b160b13c-f3ac-472a-9e84-16ad6dbe6ce1 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:03.477037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 207/400] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:21:03.517841 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:03.575836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b160b13c-f3ac-472a-9e84-16ad6dbe6ce1 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:03.610481 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3c211-7c70-472e-b356-fa4fce69c1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:03.611119 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3c211-7c70-472e-b356-fa4fce69c1a7 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:03.611397 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3c211-7c70-472e-b356-fa4fce69c1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:03.611670 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3c211-7c70-472e-b356-fa4fce69c1a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:03.622946 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3c211-7c70-472e-b356-fa4fce69c1a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:03.627745 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-b5c3c211-7c70-472e-b356-fa4fce69c1a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:21:03.627745 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:03.627745 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:03.627745 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:03.627745 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:03.627745 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:03.627745 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:03.627745 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:03.627745 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:03.627745 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:03.627745 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:03.627745 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:03.627745 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:03.627745 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:03.627745 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:03.627745 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:03.628589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:03.628589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:03.628589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:03.628589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:03.628589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:03.628589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:03.628589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:03.628589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:03.628589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:03.628589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:03.628589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:03.628589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:03.628589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:03.628589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:03.628589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:03.628589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:03.629344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:03.629344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:03.629344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:03.629344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:03.629344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:03.629344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:03.629344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:03.629344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:03.629344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:03.629344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:03.629344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:21:03.629344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:03.629344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 194/401] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:21:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:21:03.634714 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb30ee9-5ffb-49e4-8997-8a23623fd70a None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:03.690412 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:03.739337 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb30ee9-5ffb-49e4-8997-8a23623fd70a None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:03.774588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61ade5-3531-4759-a476-9f924ad0b9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:03.775483 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61ade5-3531-4759-a476-9f924ad0b9cd None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:03.775935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61ade5-3531-4759-a476-9f924ad0b9cd None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:03.776405 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61ade5-3531-4759-a476-9f924ad0b9cd None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:03.777232 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f61ade5-3531-4759-a476-9f924ad0b9cd None admin] RBAC: Adding query filter params (name=member) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:03.777704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f61ade5-3531-4759-a476-9f924ad0b9cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:03.779797 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f61ade5-3531-4759-a476-9f924ad0b9cd None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:03.792403 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 208/402] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 1 16:21:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:03.798758 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3c211-7c70-472e-b356-fa4fce69c1a7 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:03.838098 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61ade5-3531-4759-a476-9f924ad0b9cd None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:03.935687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3c211-7c70-472e-b356-fa4fce69c1a7 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:03.941153 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:03.963291 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61ade5-3531-4759-a476-9f924ad0b9cd None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:03.984024 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c2bca-1d1b-448f-be6f-4f96c660d48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:03.984619 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:03.985839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c2bca-1d1b-448f-be6f-4f96c660d48a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:03.986280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c2bca-1d1b-448f-be6f-4f96c660d48a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:03.986689 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c2bca-1d1b-448f-be6f-4f96c660d48a None admin] PATH_INFO: `/v3/groups/c06650dbbcd04814bf81f9357cdc6ede` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:03.988067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1c2bca-1d1b-448f-be6f-4f96c660d48a None admin] RBAC: Authorizing `identity:get_group(group_id=c06650dbbcd04814bf81f9357cdc6ede)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:03.989912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1c2bca-1d1b-448f-be6f-4f96c660d48a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:03.992896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 195/403] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Sep 1 16:21:03 2022] GET /identity/v3/groups/c06650dbbcd04814bf81f9357cdc6ede => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:03.993660 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c2bca-1d1b-448f-be6f-4f96c660d48a None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:04.011589 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1b033-b8c3-45c0-a85e-04ac24cd6807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:04.012384 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1b033-b8c3-45c0-a85e-04ac24cd6807 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:04.012565 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1b033-b8c3-45c0-a85e-04ac24cd6807 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:04.012818 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1b033-b8c3-45c0-a85e-04ac24cd6807 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:04.023956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f1b033-b8c3-45c0-a85e-04ac24cd6807 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:04.029751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-04f1b033-b8c3-45c0-a85e-04ac24cd6807 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:21:04.029751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:04.029751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:04.029751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:04.029751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:04.029751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:04.029751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:04.029751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:04.029751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:04.029751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:04.029751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:04.029751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:04.029751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:04.029751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:04.029751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:04.029751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:04.030627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:04.030627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:04.030627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:04.030627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:04.030627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:04.030627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:04.030627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:04.030627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:04.030627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:04.030627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:04.030627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:04.030627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:04.030627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:04.030627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:04.030627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:04.030627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:04.031381 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:04.031381 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:04.031381 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:04.031381 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:04.031381 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:04.031381 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:04.031381 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:04.031381 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:04.031381 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:04.031381 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:04.031381 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:21:04.031381 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:04.031381 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 209/404] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 1 16:21:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:21:04.032030 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1b033-b8c3-45c0-a85e-04ac24cd6807 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:04.119414 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c2bca-1d1b-448f-be6f-4f96c660d48a None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:04.155323 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1b033-b8c3-45c0-a85e-04ac24cd6807 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:04.158983 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00fc21-7cf0-4faa-9088-55ff63e80456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:04.159859 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00fc21-7cf0-4faa-9088-55ff63e80456 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:04.160311 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00fc21-7cf0-4faa-9088-55ff63e80456 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:04.160754 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00fc21-7cf0-4faa-9088-55ff63e80456 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:04.172538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb00fc21-7cf0-4faa-9088-55ff63e80456 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:04.178067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-fb00fc21-7cf0-4faa-9088-55ff63e80456 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:21:04.178067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:04.178067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:04.178067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:04.178067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:04.178067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:04.178067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:04.178067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:04.178067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:04.178067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:04.178067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:04.178067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:04.178067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:04.178067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:04.178067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:04.178067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:04.178958 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:04.178958 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:04.178958 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:04.178958 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:04.178958 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:04.178958 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:04.178958 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:04.178958 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:04.178958 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:04.178958 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:04.178958 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:04.178958 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:04.178958 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:04.178958 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:04.178958 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:04.178958 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:04.179718 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:04.179718 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:04.179718 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:04.179718 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:04.179718 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:04.179718 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:04.179718 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:04.179718 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:04.179718 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:04.179718 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:04.179718 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:21:04.179718 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:04.187238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 196/405] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:21:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:04.188320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00fc21-7cf0-4faa-9088-55ff63e80456 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:04.194786 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36e9277-d831-4153-99e7-77bdfd486324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:04.195424 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e9277-d831-4153-99e7-77bdfd486324 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:04.195717 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e9277-d831-4153-99e7-77bdfd486324 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:04.195982 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e9277-d831-4153-99e7-77bdfd486324 None admin] PATH_INFO: `/v3/projects/cc5b99bf5631484eb25353883854e856` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:04.197099 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36e9277-d831-4153-99e7-77bdfd486324 None admin] RBAC: Authorizing `identity:get_project(project_id=cc5b99bf5631484eb25353883854e856)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:04.198561 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36e9277-d831-4153-99e7-77bdfd486324 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:04.200791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 210/406] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:21:04 2022] GET /identity/v3/projects/cc5b99bf5631484eb25353883854e856 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:04.201532 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36e9277-d831-4153-99e7-77bdfd486324 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:04.223986 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:04.258741 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:04.302497 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00fc21-7cf0-4faa-9088-55ff63e80456 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:04.316656 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36e9277-d831-4153-99e7-77bdfd486324 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:04.339817 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873dd079-af08-4908-b882-c06054d06b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:04.341283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873dd079-af08-4908-b882-c06054d06b19 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:04.341734 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873dd079-af08-4908-b882-c06054d06b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:04.342173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873dd079-af08-4908-b882-c06054d06b19 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:04.343064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873dd079-af08-4908-b882-c06054d06b19 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:04.343615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873dd079-af08-4908-b882-c06054d06b19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:04.345555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873dd079-af08-4908-b882-c06054d06b19 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:04.354517 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 197/407] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:04.355598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873dd079-af08-4908-b882-c06054d06b19 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:04.374614 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ce7cc-25e2-4673-a070-264ee57cb509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:04.375441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ce7cc-25e2-4673-a070-264ee57cb509 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:04.375968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ce7cc-25e2-4673-a070-264ee57cb509 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:04.376384 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ce7cc-25e2-4673-a070-264ee57cb509 None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:04.377293 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ce7cc-25e2-4673-a070-264ee57cb509 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:04.377741 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ce7cc-25e2-4673-a070-264ee57cb509 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:04.379363 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ce7cc-25e2-4673-a070-264ee57cb509 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:04.389968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 211/408] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:21:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:04.391118 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ce7cc-25e2-4673-a070-264ee57cb509 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:04.461387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:04.471045 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873dd079-af08-4908-b882-c06054d06b19 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:04.502987 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ce7cc-25e2-4673-a070-264ee57cb509 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:04.507264 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565324e-59cb-4b12-b99a-856c5e0f6eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:04.508383 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565324e-59cb-4b12-b99a-856c5e0f6eae None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:04.508383 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565324e-59cb-4b12-b99a-856c5e0f6eae None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:04.508516 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565324e-59cb-4b12-b99a-856c5e0f6eae None admin] PATH_INFO: `/v3/projects/cc5b99bf5631484eb25353883854e856/groups/c06650dbbcd04814bf81f9357cdc6ede/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:04.510384 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565324e-59cb-4b12-b99a-856c5e0f6eae None admin] RBAC: Authorizing `identity:create_grant(project_id=cc5b99bf5631484eb25353883854e856, group_id=c06650dbbcd04814bf81f9357cdc6ede, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:04.511875 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565324e-59cb-4b12-b99a-856c5e0f6eae None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:04.513736 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:04.531509 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 198/409] 10.222.0.18 () {58 vars in 1442 bytes} [Thu Sep 1 16:21:04 2022] PUT /identity/v3/projects/cc5b99bf5631484eb25353883854e856/groups/c06650dbbcd04814bf81f9357cdc6ede/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:21:04.532651 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565324e-59cb-4b12-b99a-856c5e0f6eae None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:04.546086 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408d15e-6ec1-4dfc-9711-058e6c91406e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:04.546959 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408d15e-6ec1-4dfc-9711-058e6c91406e None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:04.547470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408d15e-6ec1-4dfc-9711-058e6c91406e None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:04.547749 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408d15e-6ec1-4dfc-9711-058e6c91406e None admin] PATH_INFO: `/v3/groups/c06650dbbcd04814bf81f9357cdc6ede` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:04.549408 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0408d15e-6ec1-4dfc-9711-058e6c91406e None admin] RBAC: Authorizing `identity:get_group(group_id=c06650dbbcd04814bf81f9357cdc6ede)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:04.551113 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0408d15e-6ec1-4dfc-9711-058e6c91406e None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:04.552922 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 212/410] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Sep 1 16:21:04 2022] GET /identity/v3/groups/c06650dbbcd04814bf81f9357cdc6ede => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:04.560033 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408d15e-6ec1-4dfc-9711-058e6c91406e None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:04.664885 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565324e-59cb-4b12-b99a-856c5e0f6eae None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:04.701998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408d15e-6ec1-4dfc-9711-058e6c91406e None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:04.702935 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915f3c5-0a6c-4b6f-a385-88137b407e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:04.703802 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915f3c5-0a6c-4b6f-a385-88137b407e69 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:04.704345 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915f3c5-0a6c-4b6f-a385-88137b407e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:04.704873 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915f3c5-0a6c-4b6f-a385-88137b407e69 None admin] PATH_INFO: `/v3/groups/c486df90c83c4eca9b47e8c3dc2bd0f6` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:04.706282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f915f3c5-0a6c-4b6f-a385-88137b407e69 None admin] RBAC: Authorizing `identity:get_group(group_id=c486df90c83c4eca9b47e8c3dc2bd0f6)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:04.707918 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f915f3c5-0a6c-4b6f-a385-88137b407e69 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:04.710038 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 199/411] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Sep 1 16:21:04 2022] GET /identity/v3/groups/c486df90c83c4eca9b47e8c3dc2bd0f6 => generated 229 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:04.715817 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915f3c5-0a6c-4b6f-a385-88137b407e69 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:04.759427 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:04.769976 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650bab1-d120-4da7-adb3-820e7bccec75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:04.770720 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650bab1-d120-4da7-adb3-820e7bccec75 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:04.771078 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650bab1-d120-4da7-adb3-820e7bccec75 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:04.771415 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650bab1-d120-4da7-adb3-820e7bccec75 None admin] PATH_INFO: `/v3/projects/cc5b99bf5631484eb25353883854e856` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:04.772585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650bab1-d120-4da7-adb3-820e7bccec75 None admin] RBAC: Authorizing `identity:get_project(project_id=cc5b99bf5631484eb25353883854e856)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:04.774101 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650bab1-d120-4da7-adb3-820e7bccec75 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:04.776009 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 213/412] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:21:04 2022] GET /identity/v3/projects/cc5b99bf5631484eb25353883854e856 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:21:04.781371 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650bab1-d120-4da7-adb3-820e7bccec75 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:04.829350 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:04.877284 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915f3c5-0a6c-4b6f-a385-88137b407e69 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:04.894778 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650bab1-d120-4da7-adb3-820e7bccec75 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:04.919936 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f77efa-152d-4995-a545-c4d2aeb77f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:04.920868 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f77efa-152d-4995-a545-c4d2aeb77f35 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:04.921571 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f77efa-152d-4995-a545-c4d2aeb77f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:04.922021 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f77efa-152d-4995-a545-c4d2aeb77f35 None admin] PATH_INFO: `/v3/projects/0d95c560fa484406b7dcc65daad0e99e` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:04.923537 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f77efa-152d-4995-a545-c4d2aeb77f35 None admin] RBAC: Authorizing `identity:get_project(project_id=0d95c560fa484406b7dcc65daad0e99e)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:04.925607 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f77efa-152d-4995-a545-c4d2aeb77f35 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:04.927658 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 200/413] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:21:04 2022] GET /identity/v3/projects/0d95c560fa484406b7dcc65daad0e99e => generated 344 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:04.933293 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f77efa-152d-4995-a545-c4d2aeb77f35 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:04.983275 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3329a690-69fb-45a6-ab72-4c1c524a5333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:04.983951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3329a690-69fb-45a6-ab72-4c1c524a5333 None admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:04.984231 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3329a690-69fb-45a6-ab72-4c1c524a5333 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:04.984505 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3329a690-69fb-45a6-ab72-4c1c524a5333 None admin] PATH_INFO: `/v3/projects/cc5b99bf5631484eb25353883854e856/groups/c06650dbbcd04814bf81f9357cdc6ede/roles/f2a6e936aa294323ba7087bf377276b2` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:04.986880 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3329a690-69fb-45a6-ab72-4c1c524a5333 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc5b99bf5631484eb25353883854e856, group_id=c06650dbbcd04814bf81f9357cdc6ede, role_id=f2a6e936aa294323ba7087bf377276b2)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:04.988335 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3329a690-69fb-45a6-ab72-4c1c524a5333 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:05.004263 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 214/414] 10.222.0.18 () {58 vars in 1442 bytes} [Thu Sep 1 16:21:04 2022] PUT /identity/v3/projects/cc5b99bf5631484eb25353883854e856/groups/c06650dbbcd04814bf81f9357cdc6ede/roles/f2a6e936aa294323ba7087bf377276b2 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:21:05.005701 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd587cd-8129-4e28-931a-2d78ff0eb013 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:05.006007 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd587cd-8129-4e28-931a-2d78ff0eb013 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:05.006305 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd587cd-8129-4e28-931a-2d78ff0eb013 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:05.007930 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 215/415] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:05.013966 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9a1ab-cf27-42c0-a99c-137843c23aff None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:05.014363 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9a1ab-cf27-42c0-a99c-137843c23aff None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:05.014612 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9a1ab-cf27-42c0-a99c-137843c23aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:05.035805 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-72d9a1ab-cf27-42c0-a99c-137843c23aff None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:05.037056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-72d9a1ab-cf27-42c0-a99c-137843c23aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:05.041964 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:05.056450 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f77efa-152d-4995-a545-c4d2aeb77f35 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:05.093408 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9ee87-cb8e-458a-9bf9-e3224b396b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:05.094182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9ee87-cb8e-458a-9bf9-e3224b396b1c None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:05.094561 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9ee87-cb8e-458a-9bf9-e3224b396b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:05.094906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9ee87-cb8e-458a-9bf9-e3224b396b1c None admin] PATH_INFO: `/v3/projects/0d95c560fa484406b7dcc65daad0e99e/groups/c486df90c83c4eca9b47e8c3dc2bd0f6/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:05.097452 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a9ee87-cb8e-458a-9bf9-e3224b396b1c None admin] RBAC: Authorizing `identity:create_grant(project_id=0d95c560fa484406b7dcc65daad0e99e, group_id=c486df90c83c4eca9b47e8c3dc2bd0f6, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:05.098791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a9ee87-cb8e-458a-9bf9-e3224b396b1c None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:05.117183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 201/416] 10.222.0.18 () {58 vars in 1442 bytes} [Thu Sep 1 16:21:04 2022] PUT /identity/v3/projects/0d95c560fa484406b7dcc65daad0e99e/groups/c486df90c83c4eca9b47e8c3dc2bd0f6/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:21:05.125778 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 216/417] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:21:05.447270 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9ee87-cb8e-458a-9bf9-e3224b396b1c None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:05.561638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9ee87-cb8e-458a-9bf9-e3224b396b1c None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:05.601304 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac30cb-5f88-4d7f-9ea0-3fc46ef7a987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:05.601956 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac30cb-5f88-4d7f-9ea0-3fc46ef7a987 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:05.602262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac30cb-5f88-4d7f-9ea0-3fc46ef7a987 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:05.602536 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac30cb-5f88-4d7f-9ea0-3fc46ef7a987 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:05.619528 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ac30cb-5f88-4d7f-9ea0-3fc46ef7a987 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:05.625107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-c0ac30cb-5f88-4d7f-9ea0-3fc46ef7a987 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:21:05.625107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:05.625107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:05.625107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:05.625107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:05.625107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:05.625107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:05.625107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:05.625107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:05.625107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:05.625107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:05.625107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:05.625107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:05.625107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:05.625107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:05.625107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:05.625970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:05.625970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:05.625970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:05.625970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:05.625970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:05.625970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:05.625970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:05.625970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:05.625970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:05.625970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:05.625970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:05.625970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:05.625970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:05.625970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:05.625970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:05.625970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:05.626763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:05.626763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:05.626763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:05.626763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:05.626763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:05.626763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:05.626763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:05.626763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:05.626763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:05.626763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:05.626763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:21:05.626763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:05.626763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 202/418] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:21:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:05.632200 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d9a1ab-cf27-42c0-a99c-137843c23aff None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:05.639731 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec41f4-dbf7-416e-b183-8c43e4e4d3d0 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:05.640021 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec41f4-dbf7-416e-b183-8c43e4e4d3d0 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:05.640303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec41f4-dbf7-416e-b183-8c43e4e4d3d0 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:05.641812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 203/419] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:21:05.646800 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c7622-1c48-48b3-8864-71cf884a0f29 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:05.647089 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c7622-1c48-48b3-8864-71cf884a0f29 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:05.647369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c7622-1c48-48b3-8864-71cf884a0f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:05.667392 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:05.669080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-263c7622-1c48-48b3-8864-71cf884a0f29 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:05.670457 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-263c7622-1c48-48b3-8864-71cf884a0f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:05.673437 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:05.751291 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d9a1ab-cf27-42c0-a99c-137843c23aff None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:05.781662 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 204/420] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:21:05.813593 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a8da6-2c85-4a44-8eed-d63569d56ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:05.814203 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a8da6-2c85-4a44-8eed-d63569d56ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:05.814503 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a8da6-2c85-4a44-8eed-d63569d56ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:05.814837 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a8da6-2c85-4a44-8eed-d63569d56ab1 None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:05.815504 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a8da6-2c85-4a44-8eed-d63569d56ab1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:05.815852 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a8da6-2c85-4a44-8eed-d63569d56ab1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:05.817233 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a8da6-2c85-4a44-8eed-d63569d56ab1 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:05.826987 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 217/421] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 1 16:21:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:05.832412 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c7622-1c48-48b3-8864-71cf884a0f29 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:05.977435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c7622-1c48-48b3-8864-71cf884a0f29 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:06.036658 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba100c5-e0e0-4af9-b708-0d746a78ec93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:06.037272 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba100c5-e0e0-4af9-b708-0d746a78ec93 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:06.037598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba100c5-e0e0-4af9-b708-0d746a78ec93 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:06.037872 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba100c5-e0e0-4af9-b708-0d746a78ec93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:06.046149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:06.121494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba100c5-e0e0-4af9-b708-0d746a78ec93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:06.123721 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba100c5-e0e0-4af9-b708-0d746a78ec93 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:06.240094 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 205/422] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:21:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 16:21:06.245616 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a8da6-2c85-4a44-8eed-d63569d56ab1 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:06.335996 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba100c5-e0e0-4af9-b708-0d746a78ec93 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:06.351718 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:06.359792 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a8da6-2c85-4a44-8eed-d63569d56ab1 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:06.396337 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cfcfe6-dbf5-4755-bb8b-62808985083e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:06.396965 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cfcfe6-dbf5-4755-bb8b-62808985083e None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:06.397242 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cfcfe6-dbf5-4755-bb8b-62808985083e None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:06.397512 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cfcfe6-dbf5-4755-bb8b-62808985083e None admin] PATH_INFO: `/v3/projects/a3f472f7ed974397b7020b42b4f22bd3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:06.398662 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cfcfe6-dbf5-4755-bb8b-62808985083e None admin] RBAC: Authorizing `identity:get_project(project_id=a3f472f7ed974397b7020b42b4f22bd3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:06.400114 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cfcfe6-dbf5-4755-bb8b-62808985083e None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:06.402039 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 218/423] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:21:06 2022] GET /identity/v3/projects/a3f472f7ed974397b7020b42b4f22bd3 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:06.407320 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cfcfe6-dbf5-4755-bb8b-62808985083e None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:06.412826 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:06.520796 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cfcfe6-dbf5-4755-bb8b-62808985083e None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:06.557988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2ae1c-3fc8-4798-9925-c652bb2cda76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:06.558646 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2ae1c-3fc8-4798-9925-c652bb2cda76 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:06.558924 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2ae1c-3fc8-4798-9925-c652bb2cda76 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:06.559340 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2ae1c-3fc8-4798-9925-c652bb2cda76 None admin] PATH_INFO: `/v3/groups/c06650dbbcd04814bf81f9357cdc6ede` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:06.560993 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb2ae1c-3fc8-4798-9925-c652bb2cda76 None admin] RBAC: Authorizing `identity:get_group(group_id=c06650dbbcd04814bf81f9357cdc6ede)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:06.562453 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb2ae1c-3fc8-4798-9925-c652bb2cda76 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:06.564008 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 219/424] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Sep 1 16:21:06 2022] GET /identity/v3/groups/c06650dbbcd04814bf81f9357cdc6ede => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:06.569626 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2ae1c-3fc8-4798-9925-c652bb2cda76 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:06.572088 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba100c5-e0e0-4af9-b708-0d746a78ec93 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:06.600737 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:06.651847 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b33990-84e9-4f11-8ae7-ea0454b7f175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:06.652488 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b33990-84e9-4f11-8ae7-ea0454b7f175 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:06.652769 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b33990-84e9-4f11-8ae7-ea0454b7f175 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:06.653074 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b33990-84e9-4f11-8ae7-ea0454b7f175 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:06.671704 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b33990-84e9-4f11-8ae7-ea0454b7f175 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:06.674158 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2ae1c-3fc8-4798-9925-c652bb2cda76 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:06.677435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-d4b33990-84e9-4f11-8ae7-ea0454b7f175 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:21:06.677435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:06.677435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:06.677435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:06.677435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:06.677435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:06.677435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:06.677435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:06.677435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:06.677435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:06.677435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:06.677435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:06.677435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:06.677435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:06.677435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:06.677435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:06.678356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:06.678356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:06.678356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:06.678356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:06.678356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:06.678356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:06.678356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:06.678356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:06.678356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:06.678356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:06.678356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:06.678356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:06.678356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:06.678356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:06.678356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:06.678356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:06.679161 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:06.679161 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:06.679161 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:06.679161 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:06.679161 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:06.679161 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:06.679161 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:06.679161 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:06.679161 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:06.679161 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:06.679161 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:21:06.679161 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:06.681818 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 206/425] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 1 16:21:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:21:06.687610 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b33990-84e9-4f11-8ae7-ea0454b7f175 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:06.719793 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d9cbf-bbd6-4d9c-a879-41f68ab9e694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:06.721430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d9cbf-bbd6-4d9c-a879-41f68ab9e694 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:06.721539 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d9cbf-bbd6-4d9c-a879-41f68ab9e694 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:06.721809 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d9cbf-bbd6-4d9c-a879-41f68ab9e694 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:06.722573 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d9cbf-bbd6-4d9c-a879-41f68ab9e694 None admin] RBAC: Adding query filter params (group.id=c06650dbbcd04814bf81f9357cdc6ede, role.id=45545334e1fc473c9d85fc91a9b0a35c, scope.project.id=a3f472f7ed974397b7020b42b4f22bd3) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:06.722881 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d9cbf-bbd6-4d9c-a879-41f68ab9e694 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:06.725341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d9cbf-bbd6-4d9c-a879-41f68ab9e694 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:06.739379 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 220/426] 10.222.0.18 () {56 vars in 1376 bytes} [Thu Sep 1 16:21:06 2022] GET /identity/v3/role_assignments?group.id=c06650dbbcd04814bf81f9357cdc6ede&scope.project.id=a3f472f7ed974397b7020b42b4f22bd3&role.id=45545334e1fc473c9d85fc91a9b0a35c => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:21:06.849411 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:06.854915 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b33990-84e9-4f11-8ae7-ea0454b7f175 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:06.900803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d360f0-7b74-4d0f-9e05-cbed7a5407ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:06.901033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d360f0-7b74-4d0f-9e05-cbed7a5407ef None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:06.901273 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d360f0-7b74-4d0f-9e05-cbed7a5407ef None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:06.901446 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d360f0-7b74-4d0f-9e05-cbed7a5407ef None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:06.902093 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d360f0-7b74-4d0f-9e05-cbed7a5407ef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:06.902419 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d360f0-7b74-4d0f-9e05-cbed7a5407ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:06.904339 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d360f0-7b74-4d0f-9e05-cbed7a5407ef None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:06.916570 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 207/427] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:06.923094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d9cbf-bbd6-4d9c-a879-41f68ab9e694 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:07.033512 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:07.045627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d9cbf-bbd6-4d9c-a879-41f68ab9e694 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:07.084073 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955a1e7-ba87-4c7a-8b40-15381bc0a2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:07.084693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955a1e7-ba87-4c7a-8b40-15381bc0a2b1 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:07.084978 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955a1e7-ba87-4c7a-8b40-15381bc0a2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:07.085256 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955a1e7-ba87-4c7a-8b40-15381bc0a2b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:07.143078 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f92e75-2014-4498-8dcb-2d94d22192f2 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:07.143567 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f92e75-2014-4498-8dcb-2d94d22192f2 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:07.143640 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f92e75-2014-4498-8dcb-2d94d22192f2 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:07.145177 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 208/428] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:07.154130 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c0a1f-f74a-4665-b97b-524825777198 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:07.154572 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c0a1f-f74a-4665-b97b-524825777198 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:07.154963 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c0a1f-f74a-4665-b97b-524825777198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:07.157654 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c955a1e7-ba87-4c7a-8b40-15381bc0a2b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:07.159162 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c955a1e7-ba87-4c7a-8b40-15381bc0a2b1 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:07.174658 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-ad5c0a1f-f74a-4665-b97b-524825777198 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:07.175989 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ad5c0a1f-f74a-4665-b97b-524825777198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:07.258116 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:07.262831 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 209/429] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:21:07.264820 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 221/430] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:21:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:21:07.270562 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c0a1f-f74a-4665-b97b-524825777198 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:07.335433 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cabecc-ceae-4392-8c42-2c9c50aa04e6 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:07.335784 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cabecc-ceae-4392-8c42-2c9c50aa04e6 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:07.336000 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cabecc-ceae-4392-8c42-2c9c50aa04e6 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:07.337337 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 222/431] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:21:07.343243 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975c75d-52d7-42ec-8f54-d8a7d235f3e8 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:07.343528 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975c75d-52d7-42ec-8f54-d8a7d235f3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:07.343803 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975c75d-52d7-42ec-8f54-d8a7d235f3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:07.365703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-7975c75d-52d7-42ec-8f54-d8a7d235f3e8 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:07.367028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-7975c75d-52d7-42ec-8f54-d8a7d235f3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:07.382742 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:07.394994 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c0a1f-f74a-4665-b97b-524825777198 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:07.442804 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc13ebf8-2997-4fca-bb62-0b82061d34a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:07.443779 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13ebf8-2997-4fca-bb62-0b82061d34a7 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:07.444250 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13ebf8-2997-4fca-bb62-0b82061d34a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:07.444742 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13ebf8-2997-4fca-bb62-0b82061d34a7 None admin] PATH_INFO: `/v3/projects/a3f472f7ed974397b7020b42b4f22bd3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:07.446064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc13ebf8-2997-4fca-bb62-0b82061d34a7 None admin] RBAC: Authorizing `identity:get_project(project_id=a3f472f7ed974397b7020b42b4f22bd3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:07.447724 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc13ebf8-2997-4fca-bb62-0b82061d34a7 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:07.454539 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 210/432] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:21:07 2022] GET /identity/v3/projects/a3f472f7ed974397b7020b42b4f22bd3 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:21:07.455936 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc13ebf8-2997-4fca-bb62-0b82061d34a7 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:07.459460 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 223/433] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:21:07.564406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc13ebf8-2997-4fca-bb62-0b82061d34a7 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:07.574077 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc765e95-6515-4db0-bc7e-8e953ffc3165 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:07.574410 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc765e95-6515-4db0-bc7e-8e953ffc3165 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:07.575331 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc765e95-6515-4db0-bc7e-8e953ffc3165 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:07.576920 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 224/434] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:07.581784 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc765e95-6515-4db0-bc7e-8e953ffc3165 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:07.602864 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2db5c6-e419-4bb9-b8c1-90f58694279d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:07.603446 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2db5c6-e419-4bb9-b8c1-90f58694279d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:07.603722 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2db5c6-e419-4bb9-b8c1-90f58694279d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:07.603998 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2db5c6-e419-4bb9-b8c1-90f58694279d None admin] PATH_INFO: `/v3/groups/c06650dbbcd04814bf81f9357cdc6ede` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:07.605108 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2db5c6-e419-4bb9-b8c1-90f58694279d None admin] RBAC: Authorizing `identity:get_group(group_id=c06650dbbcd04814bf81f9357cdc6ede)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:07.606603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2db5c6-e419-4bb9-b8c1-90f58694279d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:07.608452 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 211/435] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Sep 1 16:21:07 2022] GET /identity/v3/groups/c06650dbbcd04814bf81f9357cdc6ede => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:21:07.609610 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3321685-f480-430f-943d-16b1297b6831 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:07.609851 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3321685-f480-430f-943d-16b1297b6831 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:07.610119 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3321685-f480-430f-943d-16b1297b6831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:07.635380 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-c3321685-f480-430f-943d-16b1297b6831 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:07.636930 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c3321685-f480-430f-943d-16b1297b6831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:07.660297 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:07.704219 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc765e95-6515-4db0-bc7e-8e953ffc3165 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:07.714715 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 212/436] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:21:07.719094 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3321685-f480-430f-943d-16b1297b6831 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:07.735507 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:07.747303 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820203f0-7445-40e2-ad39-33789f5ccf2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:07.748147 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820203f0-7445-40e2-ad39-33789f5ccf2f None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:07.748431 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820203f0-7445-40e2-ad39-33789f5ccf2f None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:07.748705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820203f0-7445-40e2-ad39-33789f5ccf2f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:07.758954 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820203f0-7445-40e2-ad39-33789f5ccf2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:07.764279 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-820203f0-7445-40e2-ad39-33789f5ccf2f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:21:07.764279 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:07.764279 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:07.764279 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:07.764279 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:07.764279 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:07.764279 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:07.764279 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:07.764279 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:07.764279 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:07.764279 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:07.764279 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:07.764279 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:07.764279 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:07.764279 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:07.764279 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:07.765132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:07.765132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:07.765132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:07.765132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:07.765132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:07.765132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:07.765132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:07.765132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:07.765132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:07.765132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:07.765132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:07.765132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:07.765132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:07.765132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:07.765132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:07.765132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:07.765940 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:07.765940 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:07.765940 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:07.765940 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:07.765940 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:07.765940 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:07.765940 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:07.765940 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:07.765940 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:07.765940 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:07.765940 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:21:07.765940 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:07.765940 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 225/437] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:21:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:07.770704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820203f0-7445-40e2-ad39-33789f5ccf2f None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:07.825896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3321685-f480-430f-943d-16b1297b6831 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:07.860141 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc324d59-06d8-41a5-9a5d-560831e135e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:07.860989 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc324d59-06d8-41a5-9a5d-560831e135e2 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:07.861266 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc324d59-06d8-41a5-9a5d-560831e135e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:07.861535 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc324d59-06d8-41a5-9a5d-560831e135e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:07.862306 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc324d59-06d8-41a5-9a5d-560831e135e2 None admin] RBAC: Adding query filter params (group.id=c06650dbbcd04814bf81f9357cdc6ede, role.id=f2a6e936aa294323ba7087bf377276b2, scope.project.id=a3f472f7ed974397b7020b42b4f22bd3) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:07.862705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc324d59-06d8-41a5-9a5d-560831e135e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:07.864175 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc324d59-06d8-41a5-9a5d-560831e135e2 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:07.869899 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 213/438] 10.222.0.18 () {56 vars in 1376 bytes} [Thu Sep 1 16:21:07 2022] GET /identity/v3/role_assignments?group.id=c06650dbbcd04814bf81f9357cdc6ede&scope.project.id=a3f472f7ed974397b7020b42b4f22bd3&role.id=f2a6e936aa294323ba7087bf377276b2 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:07.870695 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc324d59-06d8-41a5-9a5d-560831e135e2 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:07.881410 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820203f0-7445-40e2-ad39-33789f5ccf2f None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:07.898744 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:07.931494 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9291c-8c52-4edf-bcfc-66b739d6e36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:07.932215 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9291c-8c52-4edf-bcfc-66b739d6e36c None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:07.932500 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9291c-8c52-4edf-bcfc-66b739d6e36c None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:07.932773 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9291c-8c52-4edf-bcfc-66b739d6e36c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:07.949914 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e9291c-8c52-4edf-bcfc-66b739d6e36c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:07.955957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-d2e9291c-8c52-4edf-bcfc-66b739d6e36c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:21:07.955957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:07.955957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:07.955957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:07.955957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:07.955957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:07.955957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:07.955957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:07.955957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:07.955957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:07.955957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:07.955957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:07.955957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:07.955957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:07.955957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:07.955957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:07.956899 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:07.956899 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:07.956899 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:07.956899 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:07.956899 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:07.956899 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:07.956899 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:07.956899 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:07.956899 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:07.956899 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:07.956899 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:07.956899 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:07.956899 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:07.956899 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:07.956899 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:07.956899 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:07.963026 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:07.963026 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:07.963026 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:07.963026 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:07.963026 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:07.963026 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:07.963026 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:07.963026 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:07.963026 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:07.963026 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:07.963026 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:21:07.963026 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:07.963026 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 226/439] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 1 16:21:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:07.963686 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9291c-8c52-4edf-bcfc-66b739d6e36c None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:08.005720 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc324d59-06d8-41a5-9a5d-560831e135e2 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:08.017468 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:08.049808 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3356393-8ecb-448c-9113-7e6da9b9e387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:08.050601 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3356393-8ecb-448c-9113-7e6da9b9e387 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:08.050889 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3356393-8ecb-448c-9113-7e6da9b9e387 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:08.051168 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3356393-8ecb-448c-9113-7e6da9b9e387 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:08.052281 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3356393-8ecb-448c-9113-7e6da9b9e387 None admin] RBAC: Adding query filter params (name=member) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:08.052281 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3356393-8ecb-448c-9113-7e6da9b9e387 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:08.053483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3356393-8ecb-448c-9113-7e6da9b9e387 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:08.063605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 214/440] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 1 16:21:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:08.064433 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3356393-8ecb-448c-9113-7e6da9b9e387 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:08.092134 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9291c-8c52-4edf-bcfc-66b739d6e36c None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:08.131707 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dfc719-ac7e-4bf2-beab-0fe22a23f7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:08.132318 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfc719-ac7e-4bf2-beab-0fe22a23f7b3 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:08.132594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfc719-ac7e-4bf2-beab-0fe22a23f7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:08.132862 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfc719-ac7e-4bf2-beab-0fe22a23f7b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:08.133525 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dfc719-ac7e-4bf2-beab-0fe22a23f7b3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:08.133825 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dfc719-ac7e-4bf2-beab-0fe22a23f7b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:08.135252 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dfc719-ac7e-4bf2-beab-0fe22a23f7b3 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:08.145186 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 227/441] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:08.146316 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dfc719-ac7e-4bf2-beab-0fe22a23f7b3 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:08.161568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:08.186139 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3356393-8ecb-448c-9113-7e6da9b9e387 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:08.226086 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572599f-3d62-4464-b71d-7a6c20e1db4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:08.226789 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572599f-3d62-4464-b71d-7a6c20e1db4a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:08.227074 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572599f-3d62-4464-b71d-7a6c20e1db4a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:08.227346 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572599f-3d62-4464-b71d-7a6c20e1db4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:08.237951 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9572599f-3d62-4464-b71d-7a6c20e1db4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:08.243171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-9572599f-3d62-4464-b71d-7a6c20e1db4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:21:08.243171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:08.243171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:08.243171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:08.243171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:08.243171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:08.243171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:08.243171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:08.243171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:08.243171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:08.243171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:08.243171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:08.243171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:08.243171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:08.243171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:08.243171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:08.244216 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:08.244216 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:08.244216 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:08.244216 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:08.244216 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:08.244216 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:08.244216 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:08.244216 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:08.244216 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:08.244216 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:08.244216 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:08.244216 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:08.244216 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:08.244216 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:08.244216 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:08.244216 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:08.245006 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:08.245006 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:08.245006 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:08.245006 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:08.245006 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:08.245006 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:08.245006 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:08.245006 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:08.245006 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:08.245006 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:08.245006 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:21:08.245006 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:08.245006 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 215/442] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:21:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:08.245006 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572599f-3d62-4464-b71d-7a6c20e1db4a None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:08.265442 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dfc719-ac7e-4bf2-beab-0fe22a23f7b3 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:08.272283 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:08.299614 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b200d7-2182-4b67-ab2d-8f2a03c48234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:08.300357 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b200d7-2182-4b67-ab2d-8f2a03c48234 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:08.300888 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b200d7-2182-4b67-ab2d-8f2a03c48234 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:08.300888 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b200d7-2182-4b67-ab2d-8f2a03c48234 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:08.342814 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572599f-3d62-4464-b71d-7a6c20e1db4a None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:08.368223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b200d7-2182-4b67-ab2d-8f2a03c48234 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:08.369701 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b200d7-2182-4b67-ab2d-8f2a03c48234 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:08.378949 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baf4fe1-3a6c-4b8a-bb3b-8e31a3b77924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:08.379562 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf4fe1-3a6c-4b8a-bb3b-8e31a3b77924 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:08.379841 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf4fe1-3a6c-4b8a-bb3b-8e31a3b77924 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:08.380125 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf4fe1-3a6c-4b8a-bb3b-8e31a3b77924 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:08.392764 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:08.448449 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baf4fe1-3a6c-4b8a-bb3b-8e31a3b77924 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:08.449845 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baf4fe1-3a6c-4b8a-bb3b-8e31a3b77924 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:08.467377 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 228/443] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:21:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:21:08.468206 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b200d7-2182-4b67-ab2d-8f2a03c48234 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:08.476059 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:08.550007 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 216/444] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:21:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:21:08.550910 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baf4fe1-3a6c-4b8a-bb3b-8e31a3b77924 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:08.568331 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b200d7-2182-4b67-ab2d-8f2a03c48234 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:08.593931 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:08.603259 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17539935-0198-4b1e-890b-47be85763db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:08.603857 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17539935-0198-4b1e-890b-47be85763db4 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:08.604136 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17539935-0198-4b1e-890b-47be85763db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:08.604405 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17539935-0198-4b1e-890b-47be85763db4 None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:08.605048 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17539935-0198-4b1e-890b-47be85763db4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:08.605349 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17539935-0198-4b1e-890b-47be85763db4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:08.607323 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17539935-0198-4b1e-890b-47be85763db4 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:08.615026 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 229/445] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:21:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:08.615792 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17539935-0198-4b1e-890b-47be85763db4 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:08.654260 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baf4fe1-3a6c-4b8a-bb3b-8e31a3b77924 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:08.679452 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:08.687730 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00110ce4-3128-4154-84a9-39cd807bba49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:08.688353 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00110ce4-3128-4154-84a9-39cd807bba49 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:08.688636 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00110ce4-3128-4154-84a9-39cd807bba49 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:08.688913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00110ce4-3128-4154-84a9-39cd807bba49 None admin] PATH_INFO: `/v3/projects/cc5b99bf5631484eb25353883854e856` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:08.690033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00110ce4-3128-4154-84a9-39cd807bba49 None admin] RBAC: Authorizing `identity:get_project(project_id=cc5b99bf5631484eb25353883854e856)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:08.691525 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00110ce4-3128-4154-84a9-39cd807bba49 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:08.693118 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 217/446] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:21:08 2022] GET /identity/v3/projects/cc5b99bf5631484eb25353883854e856 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:08.693871 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00110ce4-3128-4154-84a9-39cd807bba49 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:08.717427 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17539935-0198-4b1e-890b-47be85763db4 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:08.749282 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7fbb2-ce45-4712-ae7c-2333a8bbc089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:08.749901 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7fbb2-ce45-4712-ae7c-2333a8bbc089 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:08.750180 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7fbb2-ce45-4712-ae7c-2333a8bbc089 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:08.750504 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7fbb2-ce45-4712-ae7c-2333a8bbc089 None admin] PATH_INFO: `/v3/projects/cc5b99bf5631484eb25353883854e856` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:08.751609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b7fbb2-ce45-4712-ae7c-2333a8bbc089 None admin] RBAC: Authorizing `identity:get_project(project_id=cc5b99bf5631484eb25353883854e856)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:08.753033 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b7fbb2-ce45-4712-ae7c-2333a8bbc089 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:08.754662 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 230/447] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:21:08 2022] GET /identity/v3/projects/cc5b99bf5631484eb25353883854e856 => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:08.755407 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7fbb2-ce45-4712-ae7c-2333a8bbc089 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:08.791421 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00110ce4-3128-4154-84a9-39cd807bba49 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:08.801916 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:08.825279 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4d874-986b-403b-83ba-377953f8bb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:08.825880 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4d874-986b-403b-83ba-377953f8bb9e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:08.826158 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4d874-986b-403b-83ba-377953f8bb9e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:08.826451 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4d874-986b-403b-83ba-377953f8bb9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:08.852311 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7fbb2-ce45-4712-ae7c-2333a8bbc089 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:08.885449 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1b178-6bd5-4c59-b4ab-af04bd513cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:08.886072 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1b178-6bd5-4c59-b4ab-af04bd513cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:08.886394 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1b178-6bd5-4c59-b4ab-af04bd513cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:08.886668 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1b178-6bd5-4c59-b4ab-af04bd513cc2 None admin] PATH_INFO: `/v3/groups/c06650dbbcd04814bf81f9357cdc6ede` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:08.888318 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded1b178-6bd5-4c59-b4ab-af04bd513cc2 None admin] RBAC: Authorizing `identity:get_group(group_id=c06650dbbcd04814bf81f9357cdc6ede)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:08.889886 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded1b178-6bd5-4c59-b4ab-af04bd513cc2 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:08.891373 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 231/448] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Sep 1 16:21:08 2022] GET /identity/v3/groups/c06650dbbcd04814bf81f9357cdc6ede => generated 226 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:08.892130 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1b178-6bd5-4c59-b4ab-af04bd513cc2 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:08.893162 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a4d874-986b-403b-83ba-377953f8bb9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:08.894653 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a4d874-986b-403b-83ba-377953f8bb9e None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:08.895471 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:08.989369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1b178-6bd5-4c59-b4ab-af04bd513cc2 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:08.992656 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 218/449] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:21:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:21:08.993480 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4d874-986b-403b-83ba-377953f8bb9e None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:09.007264 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:09.023357 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fccb2c2-3b88-47f9-913f-2e12458d0e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:09.023982 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccb2c2-3b88-47f9-913f-2e12458d0e93 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:09.024263 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccb2c2-3b88-47f9-913f-2e12458d0e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:09.024536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccb2c2-3b88-47f9-913f-2e12458d0e93 None admin] PATH_INFO: `/v3/groups/c06650dbbcd04814bf81f9357cdc6ede` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:09.025629 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fccb2c2-3b88-47f9-913f-2e12458d0e93 None admin] RBAC: Authorizing `identity:get_group(group_id=c06650dbbcd04814bf81f9357cdc6ede)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:09.027101 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fccb2c2-3b88-47f9-913f-2e12458d0e93 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:09.028642 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 232/450] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Sep 1 16:21:08 2022] GET /identity/v3/groups/c06650dbbcd04814bf81f9357cdc6ede => generated 226 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:09.029383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fccb2c2-3b88-47f9-913f-2e12458d0e93 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:09.098868 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4d874-986b-403b-83ba-377953f8bb9e None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:09.099411 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:09.129514 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fccb2c2-3b88-47f9-913f-2e12458d0e93 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:09.132964 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82fc26-55ff-4b1c-a52c-ac17ff08633e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:09.133573 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82fc26-55ff-4b1c-a52c-ac17ff08633e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:09.134001 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82fc26-55ff-4b1c-a52c-ac17ff08633e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:09.134308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82fc26-55ff-4b1c-a52c-ac17ff08633e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:09.135034 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82fc26-55ff-4b1c-a52c-ac17ff08633e None admin] RBAC: Adding query filter params (group.id=c06650dbbcd04814bf81f9357cdc6ede, role.id=45545334e1fc473c9d85fc91a9b0a35c, scope.project.id=cc5b99bf5631484eb25353883854e856) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:09.135347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82fc26-55ff-4b1c-a52c-ac17ff08633e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:09.136772 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82fc26-55ff-4b1c-a52c-ac17ff08633e None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:09.143123 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 219/451] 10.222.0.18 () {56 vars in 1376 bytes} [Thu Sep 1 16:21:08 2022] GET /identity/v3/role_assignments?group.id=c06650dbbcd04814bf81f9357cdc6ede&scope.project.id=cc5b99bf5631484eb25353883854e856&role.id=45545334e1fc473c9d85fc91a9b0a35c => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:09.143921 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82fc26-55ff-4b1c-a52c-ac17ff08633e None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:09.164183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901bd77-fc4c-4df8-8646-fcc44cbf6544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:09.164839 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901bd77-fc4c-4df8-8646-fcc44cbf6544 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:09.165123 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901bd77-fc4c-4df8-8646-fcc44cbf6544 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:09.165398 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901bd77-fc4c-4df8-8646-fcc44cbf6544 None admin] PATH_INFO: `/v3/projects/0d95c560fa484406b7dcc65daad0e99e` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:09.166642 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901bd77-fc4c-4df8-8646-fcc44cbf6544 None admin] RBAC: Authorizing `identity:get_project(project_id=0d95c560fa484406b7dcc65daad0e99e)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:09.168166 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901bd77-fc4c-4df8-8646-fcc44cbf6544 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:09.169958 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 233/452] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:21:09 2022] GET /identity/v3/projects/0d95c560fa484406b7dcc65daad0e99e => generated 344 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:09.175717 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901bd77-fc4c-4df8-8646-fcc44cbf6544 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:09.255323 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:09.283439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82fc26-55ff-4b1c-a52c-ac17ff08633e None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:09.296494 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901bd77-fc4c-4df8-8646-fcc44cbf6544 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:09.327538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1836e-da0d-4cd8-988f-1a1192de4719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:09.328164 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1836e-da0d-4cd8-988f-1a1192de4719 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:09.328992 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1836e-da0d-4cd8-988f-1a1192de4719 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:09.328992 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1836e-da0d-4cd8-988f-1a1192de4719 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:09.329442 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add1836e-da0d-4cd8-988f-1a1192de4719 None admin] RBAC: Adding query filter params (group.id=c06650dbbcd04814bf81f9357cdc6ede, role.id=f2a6e936aa294323ba7087bf377276b2, scope.project.id=cc5b99bf5631484eb25353883854e856) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:09.329757 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add1836e-da0d-4cd8-988f-1a1192de4719 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:09.331362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add1836e-da0d-4cd8-988f-1a1192de4719 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:09.334912 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba32070-548f-4648-ac4e-83ae35539100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:09.335545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba32070-548f-4648-ac4e-83ae35539100 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:09.335825 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba32070-548f-4648-ac4e-83ae35539100 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:09.336105 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba32070-548f-4648-ac4e-83ae35539100 None admin] PATH_INFO: `/v3/groups/c486df90c83c4eca9b47e8c3dc2bd0f6` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:09.337218 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba32070-548f-4648-ac4e-83ae35539100 None admin] RBAC: Authorizing `identity:get_group(group_id=c486df90c83c4eca9b47e8c3dc2bd0f6)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:09.338400 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 220/453] 10.222.0.18 () {56 vars in 1376 bytes} [Thu Sep 1 16:21:09 2022] GET /identity/v3/role_assignments?group.id=c06650dbbcd04814bf81f9357cdc6ede&scope.project.id=cc5b99bf5631484eb25353883854e856&role.id=f2a6e936aa294323ba7087bf377276b2 => generated 611 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:09.338773 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba32070-548f-4648-ac4e-83ae35539100 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:09.340420 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 234/454] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Sep 1 16:21:09 2022] GET /identity/v3/groups/c486df90c83c4eca9b47e8c3dc2bd0f6 => generated 229 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:21:09.346537 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1836e-da0d-4cd8-988f-1a1192de4719 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:09.467018 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1836e-da0d-4cd8-988f-1a1192de4719 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:09.505770 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2753636-eb82-4fca-a465-c6e5de71d38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:09.506348 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2753636-eb82-4fca-a465-c6e5de71d38a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:09.506635 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2753636-eb82-4fca-a465-c6e5de71d38a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:09.507026 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2753636-eb82-4fca-a465-c6e5de71d38a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:09.507642 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2753636-eb82-4fca-a465-c6e5de71d38a None admin] RBAC: Adding query filter params (group.id=c486df90c83c4eca9b47e8c3dc2bd0f6, role.id=e8ba063905484d22993bcf04830091a3, scope.project.id=0d95c560fa484406b7dcc65daad0e99e) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:09.507956 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2753636-eb82-4fca-a465-c6e5de71d38a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:09.509404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2753636-eb82-4fca-a465-c6e5de71d38a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:09.515849 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 221/455] 10.222.0.18 () {56 vars in 1376 bytes} [Thu Sep 1 16:21:09 2022] GET /identity/v3/role_assignments?group.id=c486df90c83c4eca9b47e8c3dc2bd0f6&scope.project.id=0d95c560fa484406b7dcc65daad0e99e&role.id=e8ba063905484d22993bcf04830091a3 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:21:11.898756 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816d11c-3461-4f49-955a-b07507d2a7b1 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:11.899645 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816d11c-3461-4f49-955a-b07507d2a7b1 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:11.900062 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816d11c-3461-4f49-955a-b07507d2a7b1 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:11.901744 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 235/456] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:11.907749 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89b057-d34d-4cd9-b841-79193588c1e6 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:11.908168 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89b057-d34d-4cd9-b841-79193588c1e6 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:11.908313 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89b057-d34d-4cd9-b841-79193588c1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:11.917733 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:11.935263 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-4c89b057-d34d-4cd9-b841-79193588c1e6 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:11.936494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-4c89b057-d34d-4cd9-b841-79193588c1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:12.024850 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 222/457] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:21:12.066071 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf62ce-a1ca-4bca-9685-fc5527d08464 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:12.066762 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf62ce-a1ca-4bca-9685-fc5527d08464 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:12.067234 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf62ce-a1ca-4bca-9685-fc5527d08464 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:12.069023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 236/458] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:12.073987 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf32061-7999-400c-9888-2ded181d77c9 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:12.074309 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf32061-7999-400c-9888-2ded181d77c9 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:12.075149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf32061-7999-400c-9888-2ded181d77c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:12.099272 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-7cf32061-7999-400c-9888-2ded181d77c9 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:12.100476 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7cf32061-7999-400c-9888-2ded181d77c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:12.177150 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209dfa86-f9de-4e34-8921-c14aa1136730 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:12.177332 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209dfa86-f9de-4e34-8921-c14aa1136730 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:12.177619 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209dfa86-f9de-4e34-8921-c14aa1136730 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:12.179860 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 237/459] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:12.184889 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0171e-49ce-448c-8c64-02cf102adc03 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:12.185251 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0171e-49ce-448c-8c64-02cf102adc03 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:12.185606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0171e-49ce-448c-8c64-02cf102adc03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:12.190921 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 223/460] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:21:12.223196 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-c5e0171e-49ce-448c-8c64-02cf102adc03 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:12.224412 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-c5e0171e-49ce-448c-8c64-02cf102adc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:12.226432 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:12.320494 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 238/461] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:21:12.442694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf32061-7999-400c-9888-2ded181d77c9 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:12.460825 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e0171e-49ce-448c-8c64-02cf102adc03 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:12.514199 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:12.556685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf32061-7999-400c-9888-2ded181d77c9 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:12.593948 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543f4f9-8b3b-4163-bd4e-66c9392d4e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:12.594629 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543f4f9-8b3b-4163-bd4e-66c9392d4e85 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:12.594909 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543f4f9-8b3b-4163-bd4e-66c9392d4e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:12.595183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543f4f9-8b3b-4163-bd4e-66c9392d4e85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:12.596089 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e0171e-49ce-448c-8c64-02cf102adc03 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:12.607109 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4543f4f9-8b3b-4163-bd4e-66c9392d4e85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:12.608558 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4543f4f9-8b3b-4163-bd4e-66c9392d4e85 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:12.624559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-4543f4f9-8b3b-4163-bd4e-66c9392d4e85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:12.624559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:12.624559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:12.624559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:12.624559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:12.624559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:12.624559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:12.624559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:12.624559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:12.624559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:12.624559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:12.624559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:12.624559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:12.624559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:12.624559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:12.624559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:12.625466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:12.625466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:12.625466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:12.625466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:12.625466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:12.625466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:12.625466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:12.625466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:12.625466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:12.625466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:12.625466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:12.625466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:12.625466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:12.625466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:12.625466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:12.625466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:12.625466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:12.626309 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:12.626309 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:12.626309 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:12.630646 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 224/462] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:12.631590 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686aadfc-1719-4b57-9d1f-1622b461d5bb None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:12.631905 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686aadfc-1719-4b57-9d1f-1622b461d5bb None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:12.632162 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686aadfc-1719-4b57-9d1f-1622b461d5bb None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:12.637478 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 225/463] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:12.637478 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686aadfc-1719-4b57-9d1f-1622b461d5bb None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:12.648032 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf3d87-a663-4590-8e0a-07fa2d40090c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:12.649412 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf3d87-a663-4590-8e0a-07fa2d40090c None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:12.649891 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf3d87-a663-4590-8e0a-07fa2d40090c None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:12.650418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf3d87-a663-4590-8e0a-07fa2d40090c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:12.659559 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:12.664930 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bf3d87-a663-4590-8e0a-07fa2d40090c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:12.666507 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bf3d87-a663-4590-8e0a-07fa2d40090c None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:12.683028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-30bf3d87-a663-4590-8e0a-07fa2d40090c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:12.683028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:12.683028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:12.683028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:12.683028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:12.683028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:12.683028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:12.683028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:12.683028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:12.683028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:12.683028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:12.683028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:12.683028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:12.683028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:12.683028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:12.683028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:12.684565 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:12.684565 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:12.684565 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:12.684565 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:12.684565 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:12.684565 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:12.684565 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:12.684565 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:12.684565 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:12.684565 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:12.684565 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:12.684565 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:12.684565 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:12.684565 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:12.684565 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:12.684565 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:12.684565 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:12.685385 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:12.685385 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:12.685385 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:12.685385 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 239/464] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:12.686048 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51966723-d28e-4d0c-986f-604f846b9c54 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:12.686356 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51966723-d28e-4d0c-986f-604f846b9c54 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:12.686632 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51966723-d28e-4d0c-986f-604f846b9c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:12.706443 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-51966723-d28e-4d0c-986f-604f846b9c54 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:12.707790 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-51966723-d28e-4d0c-986f-604f846b9c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:12.754298 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686aadfc-1719-4b57-9d1f-1622b461d5bb None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:12.768150 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:12.776091 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 240/465] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:21:12.776913 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51966723-d28e-4d0c-986f-604f846b9c54 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:12.806199 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae3118-0be5-42c7-bb0b-e5be65423aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:12.807411 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae3118-0be5-42c7-bb0b-e5be65423aac None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:12.808143 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae3118-0be5-42c7-bb0b-e5be65423aac None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:12.809021 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae3118-0be5-42c7-bb0b-e5be65423aac None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:12.810134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae3118-0be5-42c7-bb0b-e5be65423aac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:12.810812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae3118-0be5-42c7-bb0b-e5be65423aac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:12.813435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae3118-0be5-42c7-bb0b-e5be65423aac None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:12.828925 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 226/466] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:12.830029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae3118-0be5-42c7-bb0b-e5be65423aac None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:12.890816 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51966723-d28e-4d0c-986f-604f846b9c54 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:12.912162 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:12.928518 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b86ec-cb53-4a92-bc48-c13ebf08abc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:12.929153 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b86ec-cb53-4a92-bc48-c13ebf08abc0 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:12.929432 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b86ec-cb53-4a92-bc48-c13ebf08abc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:12.929704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b86ec-cb53-4a92-bc48-c13ebf08abc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:12.940586 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b86ec-cb53-4a92-bc48-c13ebf08abc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:12.942322 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b86ec-cb53-4a92-bc48-c13ebf08abc0 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:12.957572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-b95b86ec-cb53-4a92-bc48-c13ebf08abc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:12.957572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:12.957572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:12.957572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:12.957572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:12.957572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:12.957572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:12.957572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:12.957572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:12.957572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:12.957572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:12.957572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:12.957572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:12.957572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:12.957572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:12.957572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:12.958487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:12.958487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:12.958487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:12.958487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:12.958487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:12.958487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:12.958487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:12.958487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:12.958487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:12.958487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:12.958487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:12.958487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:12.958487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:12.958487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:12.958487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:12.958487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:12.958487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:12.959329 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:12.959329 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:12.959329 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:12.959329 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 241/467] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:12.959329 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b86ec-cb53-4a92-bc48-c13ebf08abc0 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:12.961695 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae3118-0be5-42c7-bb0b-e5be65423aac None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:13.010598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79685435-1ef5-46a0-89ff-99104c21da5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:13.011724 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79685435-1ef5-46a0-89ff-99104c21da5f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:13.012207 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79685435-1ef5-46a0-89ff-99104c21da5f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:13.012875 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79685435-1ef5-46a0-89ff-99104c21da5f None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:13.014108 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79685435-1ef5-46a0-89ff-99104c21da5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:13.014585 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79685435-1ef5-46a0-89ff-99104c21da5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:13.016658 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79685435-1ef5-46a0-89ff-99104c21da5f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:13.033429 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 227/468] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:13.035408 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79685435-1ef5-46a0-89ff-99104c21da5f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:13.071485 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b86ec-cb53-4a92-bc48-c13ebf08abc0 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:13.081341 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:13.110955 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40fc02-839f-4450-b4fe-95192fdc5527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:13.111670 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40fc02-839f-4450-b4fe-95192fdc5527 None admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:13.111988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40fc02-839f-4450-b4fe-95192fdc5527 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:13.112217 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40fc02-839f-4450-b4fe-95192fdc5527 None admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:13.112858 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c40fc02-839f-4450-b4fe-95192fdc5527 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:13.114323 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c40fc02-839f-4450-b4fe-95192fdc5527 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:13.136408 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 242/469] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 1 16:21:12 2022] POST /identity/v3/users => generated 248 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:21:13.138048 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40fc02-839f-4450-b4fe-95192fdc5527 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:13.175680 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:13.183317 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79685435-1ef5-46a0-89ff-99104c21da5f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:13.235113 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0048ddf-5ba6-49af-8395-c685f60cb198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:13.235694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0048ddf-5ba6-49af-8395-c685f60cb198 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:13.235983 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0048ddf-5ba6-49af-8395-c685f60cb198 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:13.236264 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0048ddf-5ba6-49af-8395-c685f60cb198 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:13.236988 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0048ddf-5ba6-49af-8395-c685f60cb198 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:13.237301 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0048ddf-5ba6-49af-8395-c685f60cb198 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:13.238782 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0048ddf-5ba6-49af-8395-c685f60cb198 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:13.260694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 228/470] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:13.260694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0048ddf-5ba6-49af-8395-c685f60cb198 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:13.278522 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40fc02-839f-4450-b4fe-95192fdc5527 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:13.329231 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305c666-04b0-4916-a39d-d81aa49a73b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:13.330600 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305c666-04b0-4916-a39d-d81aa49a73b6 None admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:13.330600 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305c666-04b0-4916-a39d-d81aa49a73b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:13.330600 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305c666-04b0-4916-a39d-d81aa49a73b6 None admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:13.331115 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d305c666-04b0-4916-a39d-d81aa49a73b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:13.332586 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d305c666-04b0-4916-a39d-d81aa49a73b6 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:13.360299 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 243/471] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 1 16:21:13 2022] POST /identity/v3/users => generated 246 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:21:13.361508 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305c666-04b0-4916-a39d-d81aa49a73b6 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:13.392944 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:13.404876 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0048ddf-5ba6-49af-8395-c685f60cb198 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:13.485980 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:13.486936 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b4918-9208-418e-963c-289f1c5cc301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:13.487796 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b4918-9208-418e-963c-289f1c5cc301 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:13.487993 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b4918-9208-418e-963c-289f1c5cc301 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:13.488237 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b4918-9208-418e-963c-289f1c5cc301 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:13.499770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305c666-04b0-4916-a39d-d81aa49a73b6 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:13.513110 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b4918-9208-418e-963c-289f1c5cc301 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:13.521710 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b4918-9208-418e-963c-289f1c5cc301 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:13.538220 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb3a96-feed-4513-8d71-8d47cf8c0a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:13.538866 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb3a96-feed-4513-8d71-8d47cf8c0a50 None admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:13.539152 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb3a96-feed-4513-8d71-8d47cf8c0a50 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:13.539427 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb3a96-feed-4513-8d71-8d47cf8c0a50 None admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:13.540077 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eb3a96-feed-4513-8d71-8d47cf8c0a50 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:13.541537 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eb3a96-feed-4513-8d71-8d47cf8c0a50 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:13.544412 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-9e8b4918-9208-418e-963c-289f1c5cc301 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:13.544412 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:13.544412 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:13.544412 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:13.544412 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:13.544412 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:13.544412 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:13.544412 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:13.544412 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:13.544412 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:13.544412 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:13.544412 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:13.544412 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:13.544412 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:13.544412 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:13.544412 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:13.545261 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:13.545261 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:13.545261 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:13.545261 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:13.545261 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:13.545261 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:13.545261 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:13.545261 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:13.545261 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:13.545261 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:13.545261 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:13.545261 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:13.545261 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:13.545261 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:13.545261 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:13.545261 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:13.545261 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:13.546054 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:13.546054 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:13.546054 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:13.548965 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 229/472] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:13.554346 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b4918-9208-418e-963c-289f1c5cc301 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:13.569457 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 244/473] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 1 16:21:13 2022] POST /identity/v3/users => generated 248 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:21:13.571050 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57d14f-01eb-42d6-9cc6-8ff1112f3ef4 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:13.571343 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57d14f-01eb-42d6-9cc6-8ff1112f3ef4 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:13.571621 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57d14f-01eb-42d6-9cc6-8ff1112f3ef4 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:13.572987 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 245/474] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:13.579723 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8c8d2-0193-4984-a8bc-53140a791612 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:13.580071 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8c8d2-0193-4984-a8bc-53140a791612 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:13.580420 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8c8d2-0193-4984-a8bc-53140a791612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:13.601776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-20f8c8d2-0193-4984-a8bc-53140a791612 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:13.603085 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-20f8c8d2-0193-4984-a8bc-53140a791612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:13.695344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b4918-9208-418e-963c-289f1c5cc301 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:13.696027 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 246/475] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:21:13.751006 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a003725-8b2b-4481-a4ac-2d5e56cc3b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:13.751983 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a003725-8b2b-4481-a4ac-2d5e56cc3b76 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:13.752518 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a003725-8b2b-4481-a4ac-2d5e56cc3b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:13.753090 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a003725-8b2b-4481-a4ac-2d5e56cc3b76 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:13.753974 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a003725-8b2b-4481-a4ac-2d5e56cc3b76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:13.754502 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a003725-8b2b-4481-a4ac-2d5e56cc3b76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:13.756085 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a003725-8b2b-4481-a4ac-2d5e56cc3b76 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:13.770442 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 230/476] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:13.775659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8c8d2-0193-4984-a8bc-53140a791612 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:13.863258 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:13.906186 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8c8d2-0193-4984-a8bc-53140a791612 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:13.948859 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b25c3-0bd2-44a8-8288-197f5cfdeaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:13.949529 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b25c3-0bd2-44a8-8288-197f5cfdeaf2 None admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:13.949770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b25c3-0bd2-44a8-8288-197f5cfdeaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:13.950045 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b25c3-0bd2-44a8-8288-197f5cfdeaf2 None admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:13.950755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387b25c3-0bd2-44a8-8288-197f5cfdeaf2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:13.952272 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387b25c3-0bd2-44a8-8288-197f5cfdeaf2 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:13.989635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 247/477] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 1 16:21:13 2022] POST /identity/v3/users => generated 249 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:21:14.019309 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a003725-8b2b-4481-a4ac-2d5e56cc3b76 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:14.029386 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:14.154489 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a003725-8b2b-4481-a4ac-2d5e56cc3b76 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:14.197412 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb3dd60-40bc-4bf8-8a75-3c17d7ea0634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:14.198031 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3dd60-40bc-4bf8-8a75-3c17d7ea0634 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:14.198345 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3dd60-40bc-4bf8-8a75-3c17d7ea0634 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:14.198627 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3dd60-40bc-4bf8-8a75-3c17d7ea0634 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:14.199336 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb3dd60-40bc-4bf8-8a75-3c17d7ea0634 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:14.200703 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb3dd60-40bc-4bf8-8a75-3c17d7ea0634 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:14.209523 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.sql.core [None req-1cb3dd60-40bc-4bf8-8a75-3c17d7ea0634 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 16:21:14.209523 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 16:21:14.209523 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [parameters: {'id': 'e464ab4f71ae4ad3bb10ccc58d3bb0dc', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 16:21:14.209523 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81551) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 16:21:14.226603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-1cb3dd60-40bc-4bf8-8a75-3c17d7ea0634 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 16:21:14.226603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:14.226603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 16:21:14.226603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 16:21:14.226603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 16:21:14.226603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 16:21:14.226603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 16:21:14.226603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 16:21:14.226603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 16:21:14.226603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application conn.query(q) Sep 01 16:21:14.226603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 16:21:14.226603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 16:21:14.226603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 16:21:14.226603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application result.read() Sep 01 16:21:14.226603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 16:21:14.227487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 16:21:14.227487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 16:21:14.227487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 16:21:14.227487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 16:21:14.227487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 16:21:14.227487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 16:21:14.227487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 16:21:14.227487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 16:21:14.227487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:14.227487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 16:21:14.227487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:14.227487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:14.227487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 16:21:14.227487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 16:21:14.227487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 01 16:21:14.227487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return ref.to_dict() Sep 01 16:21:14.227487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 16:21:14.227487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application next(self.gen) Sep 01 16:21:14.228362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 16:21:14.228362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application yield resource Sep 01 16:21:14.228362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 16:21:14.228362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application next(self.gen) Sep 01 16:21:14.228362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 16:21:14.228362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 16:21:14.228362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 16:21:14.228362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 16:21:14.228362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 16:21:14.228362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise self.value Sep 01 16:21:14.228362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 16:21:14.228362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 16:21:14.228362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 16:21:14.228362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application session.commit() Sep 01 16:21:14.228362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 16:21:14.228362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 16:21:14.228362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 16:21:14.229187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 16:21:14.229187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 16:21:14.229187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self.session.flush() Sep 01 16:21:14.229187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 16:21:14.229187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 16:21:14.229187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 16:21:14.229187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 16:21:14.229187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 16:21:14.229187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application compat.raise_( Sep 01 16:21:14.229187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 16:21:14.229187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception Sep 01 16:21:14.229187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 16:21:14.229187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 16:21:14.229187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 16:21:14.229187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 16:21:14.229187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 16:21:14.229187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 16:21:14.230026 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 16:21:14.230026 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 16:21:14.230026 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 16:21:14.230026 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 16:21:14.230026 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 16:21:14.230026 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 16:21:14.230026 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 16:21:14.230026 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 16:21:14.230026 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 16:21:14.230026 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 16:21:14.230026 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 16:21:14.230026 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 16:21:14.230026 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 16:21:14.230026 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 16:21:14.230026 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 16:21:14.230026 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception Sep 01 16:21:14.231369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 16:21:14.231369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 16:21:14.231369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 16:21:14.231369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 16:21:14.231369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 16:21:14.231369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 16:21:14.231369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 16:21:14.231369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application conn.query(q) Sep 01 16:21:14.231369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 16:21:14.231369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 16:21:14.231369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 16:21:14.231369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application result.read() Sep 01 16:21:14.231369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 16:21:14.231369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 16:21:14.231369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 16:21:14.231369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 16:21:14.232111 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 16:21:14.232111 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 16:21:14.232111 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 16:21:14.232111 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 16:21:14.232111 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 16:21:14.232111 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 16:21:14.232111 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [parameters: {'id': 'e464ab4f71ae4ad3bb10ccc58d3bb0dc', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 16:21:14.232111 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 16:21:14.232111 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:14.232111 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:21:14.232111 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:14.232111 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:14.232111 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:14.232111 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:14.232111 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:14.232111 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:14.232862 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:14.232862 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:14.232862 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:14.232862 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:14.232862 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:14.232862 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:14.232862 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 01 16:21:14.232862 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 01 16:21:14.232862 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:14.232862 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:14.232862 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 01 16:21:14.232862 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 01 16:21:14.232862 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 16:21:14.232862 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 16:21:14.232862 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 16:21:14.232862 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:14.235211 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 231/478] 10.222.0.18 () {60 vars in 1131 bytes} [Thu Sep 1 16:21:14 2022] POST /identity/v3/roles => generated 144 bytes in 210 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:14.235365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b25c3-0bd2-44a8-8288-197f5cfdeaf2 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:14.358167 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b25c3-0bd2-44a8-8288-197f5cfdeaf2 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:14.390053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:14.402120 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb959b-f50d-469b-98eb-7717e3bccc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:14.403357 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb959b-f50d-469b-98eb-7717e3bccc89 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:14.403642 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb959b-f50d-469b-98eb-7717e3bccc89 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:14.403919 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb959b-f50d-469b-98eb-7717e3bccc89 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:14.415533 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb959b-f50d-469b-98eb-7717e3bccc89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:14.422107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-83bb959b-f50d-469b-98eb-7717e3bccc89 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:21:14.422107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:14.422107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:14.422107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:14.422107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:14.422107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:14.422107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:14.422107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:14.422107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:14.422107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:14.422107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:14.422107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:14.422107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:14.422107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:14.422107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:14.422107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:14.422968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:14.422968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:14.422968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:14.422968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:14.422968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:14.422968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:14.422968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:14.422968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:14.422968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:14.422968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:14.422968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:14.422968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:14.422968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:14.422968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:14.422968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:14.422968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:14.423757 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:14.423757 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:14.423757 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:14.423757 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:14.423757 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:14.423757 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:14.423757 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:14.423757 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:14.423757 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:14.423757 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:14.423757 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:21:14.423757 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:14.426407 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 248/479] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 1 16:21:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:14.429578 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb3dd60-40bc-4bf8-8a75-3c17d7ea0634 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:14.508984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:14.544923 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb3dd60-40bc-4bf8-8a75-3c17d7ea0634 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:14.582619 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339c7ca-2574-411f-810a-b9b29d9c4a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:14.583241 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339c7ca-2574-411f-810a-b9b29d9c4a56 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:14.583526 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339c7ca-2574-411f-810a-b9b29d9c4a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:14.583804 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339c7ca-2574-411f-810a-b9b29d9c4a56 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:14.584476 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339c7ca-2574-411f-810a-b9b29d9c4a56 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:14.584782 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339c7ca-2574-411f-810a-b9b29d9c4a56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:14.586151 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339c7ca-2574-411f-810a-b9b29d9c4a56 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:14.600524 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 232/480] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:15.530631 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5a3ea-c03b-45d4-b621-c49143432164 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:15.531340 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5a3ea-c03b-45d4-b621-c49143432164 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:15.531340 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5a3ea-c03b-45d4-b621-c49143432164 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:15.532783 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 249/481] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:15.537806 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5bec7-ebe1-4a11-8b0a-a77aefc9c45c None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:15.538319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5bec7-ebe1-4a11-8b0a-a77aefc9c45c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:15.538493 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5bec7-ebe1-4a11-8b0a-a77aefc9c45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:15.561345 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-62e5bec7-ebe1-4a11-8b0a-a77aefc9c45c None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:15.562589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-62e5bec7-ebe1-4a11-8b0a-a77aefc9c45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:15.652788 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 233/482] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:21:15.728152 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69573cab-bcce-40f1-8887-bd98c62e03ef None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:15.728479 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69573cab-bcce-40f1-8887-bd98c62e03ef None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:15.728686 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69573cab-bcce-40f1-8887-bd98c62e03ef None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:15.730282 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 250/483] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:15.735792 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c2b76-d1fd-45c2-b8f7-18ae7938edec None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:15.736089 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c2b76-d1fd-45c2-b8f7-18ae7938edec None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:15.736369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c2b76-d1fd-45c2-b8f7-18ae7938edec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:15.763160 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-048c2b76-d1fd-45c2-b8f7-18ae7938edec None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:15.764427 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-048c2b76-d1fd-45c2-b8f7-18ae7938edec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:15.839522 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:15.851672 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 234/484] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:21:15.978758 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69573cab-bcce-40f1-8887-bd98c62e03ef None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:16.102948 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69573cab-bcce-40f1-8887-bd98c62e03ef None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:16.138274 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfc2db-0a32-470d-a2a8-623ae2f5dd82 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:16.138274 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfc2db-0a32-470d-a2a8-623ae2f5dd82 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:16.138274 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfc2db-0a32-470d-a2a8-623ae2f5dd82 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:16.139688 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 235/485] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:16.145218 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38324221-10d3-461b-82b6-eaba7c6242af None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:16.145867 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38324221-10d3-461b-82b6-eaba7c6242af None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:16.146467 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38324221-10d3-461b-82b6-eaba7c6242af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:16.168389 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e57a0-5b77-45cb-8d60-5f25cd66a5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:16.169076 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e57a0-5b77-45cb-8d60-5f25cd66a5dc None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:16.169363 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e57a0-5b77-45cb-8d60-5f25cd66a5dc None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:16.169638 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e57a0-5b77-45cb-8d60-5f25cd66a5dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:16.174089 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-38324221-10d3-461b-82b6-eaba7c6242af None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:16.176636 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-38324221-10d3-461b-82b6-eaba7c6242af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:16.188032 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661e57a0-5b77-45cb-8d60-5f25cd66a5dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:16.193380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-661e57a0-5b77-45cb-8d60-5f25cd66a5dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:16.193380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:16.193380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:16.193380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:16.193380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:16.193380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:16.193380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:16.193380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:16.193380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:16.193380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:16.193380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:16.193380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:16.193380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:16.193380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:16.193380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:16.193380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:16.194225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:16.194225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:16.194225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:16.194225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:16.194225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:16.194225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:16.194225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:16.194225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:16.194225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:16.194225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:16.194225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:16.194225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:16.194225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:16.194225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:16.194225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:16.194225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:16.195043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:16.195043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:16.195043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:16.195043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:16.195043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:16.195043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:16.195043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:16.195043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:16.195043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:16.195043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:16.195043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:16.195043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:16.197601 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 251/486] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 1 16:21:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:16.199744 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e57a0-5b77-45cb-8d60-5f25cd66a5dc None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:16.250316 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:16.252726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 236/487] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:21:16.253885 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38324221-10d3-461b-82b6-eaba7c6242af None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:16.339080 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e57a0-5b77-45cb-8d60-5f25cd66a5dc None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:16.365334 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38324221-10d3-461b-82b6-eaba7c6242af None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:16.390503 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:16.401611 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a673f86-10d7-40c8-a4c6-3615954e0a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:16.405208 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a673f86-10d7-40c8-a4c6-3615954e0a50 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:16.405515 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a673f86-10d7-40c8-a4c6-3615954e0a50 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:16.405799 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a673f86-10d7-40c8-a4c6-3615954e0a50 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:16.408462 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d39c8c2-1aa7-4ac2-b57c-0ef507b2d08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:16.409075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39c8c2-1aa7-4ac2-b57c-0ef507b2d08c None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:16.409358 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39c8c2-1aa7-4ac2-b57c-0ef507b2d08c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:16.409635 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39c8c2-1aa7-4ac2-b57c-0ef507b2d08c None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:16.410327 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d39c8c2-1aa7-4ac2-b57c-0ef507b2d08c None admin] RBAC: Adding query filter params (name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:16.410636 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d39c8c2-1aa7-4ac2-b57c-0ef507b2d08c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:16.412007 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d39c8c2-1aa7-4ac2-b57c-0ef507b2d08c None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:16.428502 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a673f86-10d7-40c8-a4c6-3615954e0a50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:16.435133 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 237/488] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 1 16:21:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:16.435283 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-8a673f86-10d7-40c8-a4c6-3615954e0a50 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:16.435283 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:16.435283 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:16.435283 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:16.435283 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:16.435283 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:16.435283 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:16.435283 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:16.435283 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:16.435283 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:16.435283 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:16.435283 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:16.435283 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:16.435283 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:16.435283 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:16.435283 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:16.436230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:16.436230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:16.436230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:16.436230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:16.436230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:16.436230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:16.436230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:16.436230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:16.436230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:16.436230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:16.436230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:16.436230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:16.436230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:16.436230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:16.436230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:16.436230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:16.437011 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:16.437011 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:16.437011 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:16.437011 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:16.437011 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:16.437011 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:16.437011 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:16.437011 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:16.437011 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:16.437011 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:16.437011 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:16.437011 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:16.437011 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 252/489] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 1 16:21:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:16.442487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d39c8c2-1aa7-4ac2-b57c-0ef507b2d08c None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:16.447226 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a673f86-10d7-40c8-a4c6-3615954e0a50 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:16.557460 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d39c8c2-1aa7-4ac2-b57c-0ef507b2d08c None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:16.593962 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7f4f1-6d64-4f93-9e3e-d6174e05097b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:16.594609 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7f4f1-6d64-4f93-9e3e-d6174e05097b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:16.594898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7f4f1-6d64-4f93-9e3e-d6174e05097b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:16.595180 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7f4f1-6d64-4f93-9e3e-d6174e05097b None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:16.595845 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a7f4f1-6d64-4f93-9e3e-d6174e05097b None admin] RBAC: Adding query filter params (name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:16.596155 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a7f4f1-6d64-4f93-9e3e-d6174e05097b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:16.597519 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a7f4f1-6d64-4f93-9e3e-d6174e05097b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:16.606144 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 238/490] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 1 16:21:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:21:16.611578 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b729db4-4a45-4635-b5ac-bf56ce783135 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:16.612206 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b729db4-4a45-4635-b5ac-bf56ce783135 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:16.612485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b729db4-4a45-4635-b5ac-bf56ce783135 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:16.614136 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 239/491] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:16.614724 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b729db4-4a45-4635-b5ac-bf56ce783135 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:16.634000 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a673f86-10d7-40c8-a4c6-3615954e0a50 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:16.642052 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:16.670988 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:16.672099 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7234a57e-08e1-4b23-921f-ddb0cf6c68d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:16.672793 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234a57e-08e1-4b23-921f-ddb0cf6c68d2 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:16.673076 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234a57e-08e1-4b23-921f-ddb0cf6c68d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:16.673390 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234a57e-08e1-4b23-921f-ddb0cf6c68d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:16.684216 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7234a57e-08e1-4b23-921f-ddb0cf6c68d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:16.686436 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7234a57e-08e1-4b23-921f-ddb0cf6c68d2 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:16.705419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-7234a57e-08e1-4b23-921f-ddb0cf6c68d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:16.705419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:16.705419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:16.705419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:16.705419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:16.705419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:16.705419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:16.705419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:16.705419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:16.705419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:16.705419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:16.705419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:16.705419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:16.705419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:16.705419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:16.705419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:16.706335 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:16.706335 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:16.706335 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:16.706335 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:16.706335 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:16.706335 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:16.706335 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:16.706335 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:16.706335 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:16.706335 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:16.706335 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:16.706335 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:16.706335 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:16.706335 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:16.706335 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:16.706335 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:16.706335 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:16.707155 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:16.707155 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:16.707155 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:16.707959 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 253/492] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:16.708772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7234a57e-08e1-4b23-921f-ddb0cf6c68d2 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:16.732804 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b729db4-4a45-4635-b5ac-bf56ce783135 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:16.768935 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6768b93b-4821-4025-9154-69cca6e039b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:16.769725 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768b93b-4821-4025-9154-69cca6e039b8 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:16.770324 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768b93b-4821-4025-9154-69cca6e039b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:16.770630 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768b93b-4821-4025-9154-69cca6e039b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:16.781983 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6768b93b-4821-4025-9154-69cca6e039b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:16.787296 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-6768b93b-4821-4025-9154-69cca6e039b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:16.787296 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:16.787296 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:16.787296 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:16.787296 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:16.787296 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:16.787296 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:16.787296 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:16.787296 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:16.787296 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:16.787296 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:16.787296 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:16.787296 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:16.787296 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:16.787296 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:16.787296 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:16.788184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:16.788184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:16.788184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:16.788184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:16.788184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:16.788184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:16.788184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:16.788184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:16.788184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:16.788184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:16.788184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:16.788184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:16.788184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:16.788184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:16.788184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:16.788184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:16.788931 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:16.788931 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:16.788931 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:16.788931 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:16.788931 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:16.788931 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:16.788931 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:16.788931 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:16.788931 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:16.788931 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:16.788931 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:16.788931 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:16.788931 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 240/493] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 1 16:21:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:16.789610 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020f374-783d-4636-82af-23591c0e26a9 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:16.790316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020f374-783d-4636-82af-23591c0e26a9 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:16.790316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020f374-783d-4636-82af-23591c0e26a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:16.809257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-a020f374-783d-4636-82af-23591c0e26a9 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:16.810409 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a020f374-783d-4636-82af-23591c0e26a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:16.824667 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7234a57e-08e1-4b23-921f-ddb0cf6c68d2 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:16.870671 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b305a867-af43-45f9-814b-b84215d9d818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:16.871843 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305a867-af43-45f9-814b-b84215d9d818 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:16.872126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305a867-af43-45f9-814b-b84215d9d818 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:16.872433 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305a867-af43-45f9-814b-b84215d9d818 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:16.886397 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 241/494] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:21:16.886397 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020f374-783d-4636-82af-23591c0e26a9 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:16.888015 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b305a867-af43-45f9-814b-b84215d9d818 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:16.889551 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b305a867-af43-45f9-814b-b84215d9d818 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:16.906635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-b305a867-af43-45f9-814b-b84215d9d818 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:16.906635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:16.906635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:16.906635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:16.906635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:16.906635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:16.906635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:16.906635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:16.906635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:16.906635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:16.906635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:16.906635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:16.906635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:16.906635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:16.906635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:16.906635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:16.907525 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:16.907525 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:16.907525 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:16.907525 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:16.907525 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:16.907525 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:16.907525 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:16.907525 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:16.907525 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:16.907525 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:16.907525 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:16.907525 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:16.907525 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:16.907525 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:16.907525 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:16.907525 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:16.907525 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:16.908332 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:16.908332 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:16.908332 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:16.910089 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 254/495] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:16.911032 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b305a867-af43-45f9-814b-b84215d9d818 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:16.925846 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:16.930724 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:17.003812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020f374-783d-4636-82af-23591c0e26a9 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:17.037305 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b305a867-af43-45f9-814b-b84215d9d818 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:17.040664 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d848a-9a56-42ac-99d4-0f74218ef47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:17.041759 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d848a-9a56-42ac-99d4-0f74218ef47f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:17.042514 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d848a-9a56-42ac-99d4-0f74218ef47f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:17.043257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d848a-9a56-42ac-99d4-0f74218ef47f None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:17.044670 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d848a-9a56-42ac-99d4-0f74218ef47f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:17.045399 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d848a-9a56-42ac-99d4-0f74218ef47f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:17.047470 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d848a-9a56-42ac-99d4-0f74218ef47f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:17.060865 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 242/496] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:17.062046 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d848a-9a56-42ac-99d4-0f74218ef47f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:17.076414 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427d5a4-715c-4b63-8eb6-aa41ff95402c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:17.077038 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427d5a4-715c-4b63-8eb6-aa41ff95402c None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:17.077525 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427d5a4-715c-4b63-8eb6-aa41ff95402c None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:17.077644 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427d5a4-715c-4b63-8eb6-aa41ff95402c None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:17.078225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0427d5a4-715c-4b63-8eb6-aa41ff95402c None admin] RBAC: Adding query filter params (name=service) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:17.078553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0427d5a4-715c-4b63-8eb6-aa41ff95402c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:17.079937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0427d5a4-715c-4b63-8eb6-aa41ff95402c None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:17.093202 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 255/497] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 1 16:21:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:17.094051 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427d5a4-715c-4b63-8eb6-aa41ff95402c None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:17.166064 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:17.183607 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:17.190153 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d848a-9a56-42ac-99d4-0f74218ef47f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:17.196821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427d5a4-715c-4b63-8eb6-aa41ff95402c None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:17.225710 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004a235-73c0-4703-b31a-8f2a451b2189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:17.226356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004a235-73c0-4703-b31a-8f2a451b2189 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:17.226639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004a235-73c0-4703-b31a-8f2a451b2189 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:17.226914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004a235-73c0-4703-b31a-8f2a451b2189 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:17.227555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6004a235-73c0-4703-b31a-8f2a451b2189 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:17.227860 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6004a235-73c0-4703-b31a-8f2a451b2189 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:17.229222 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6004a235-73c0-4703-b31a-8f2a451b2189 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:17.233341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b016ebef-1f9b-434b-983a-130f1386e398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:17.234172 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016ebef-1f9b-434b-983a-130f1386e398 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:17.234652 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016ebef-1f9b-434b-983a-130f1386e398 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:17.235095 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016ebef-1f9b-434b-983a-130f1386e398 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:17.241346 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 243/498] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:17.243071 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004a235-73c0-4703-b31a-8f2a451b2189 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:17.248212 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b016ebef-1f9b-434b-983a-130f1386e398 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:17.249106 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b016ebef-1f9b-434b-983a-130f1386e398 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:17.262430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-b016ebef-1f9b-434b-983a-130f1386e398 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 16:21:17.262430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:17.262430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:17.262430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:17.262430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:17.262430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:17.262430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:17.262430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:17.262430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:17.262430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:17.262430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:17.262430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:17.262430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:21:17.262430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:21:17.262430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:21:17.262430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:21:17.263308 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:17.263308 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:17.263308 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:21:17.263308 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:17.263308 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:21:17.263308 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:17.263308 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:17.263308 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:17.263308 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:17.263308 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:17.263308 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:17.263308 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:17.263308 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:17.263308 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:17.263308 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:17.263308 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:17.263308 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:17.263308 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:17.264158 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:17.264158 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:17.264158 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:21:17.264158 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:21:17.264158 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:21:17.264158 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:21:17.264158 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:21:17.264158 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:21:17.264158 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 16:21:17.264158 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:17.264158 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 256/499] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:21:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:17.264705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b016ebef-1f9b-434b-983a-130f1386e398 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:17.348117 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004a235-73c0-4703-b31a-8f2a451b2189 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:17.384714 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720db6f0-6031-42ba-ac81-2d54dbf75bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:17.385348 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720db6f0-6031-42ba-ac81-2d54dbf75bab None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:17.385627 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720db6f0-6031-42ba-ac81-2d54dbf75bab None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:17.385905 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720db6f0-6031-42ba-ac81-2d54dbf75bab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:17.389082 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b016ebef-1f9b-434b-983a-130f1386e398 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:17.396470 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720db6f0-6031-42ba-ac81-2d54dbf75bab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:17.397913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720db6f0-6031-42ba-ac81-2d54dbf75bab None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:17.413118 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-720db6f0-6031-42ba-ac81-2d54dbf75bab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:17.413118 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:17.413118 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:17.413118 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:17.413118 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:17.413118 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:17.413118 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:17.413118 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:17.413118 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:17.413118 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:17.413118 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:17.413118 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:17.413118 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:17.413118 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:17.413118 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:17.413118 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:17.413970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:17.413970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:17.413970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:17.413970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:17.413970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:17.413970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:17.413970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:17.413970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:17.413970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:17.413970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:17.413970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:17.413970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:17.413970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:17.413970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:17.413970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:17.413970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:17.413970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:17.414813 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:17.414813 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:17.414813 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:17.414813 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 244/500] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:17.415015 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720db6f0-6031-42ba-ac81-2d54dbf75bab None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:17.421895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:17.425803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:17.427442 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec95ad5-d4a3-451e-9a2e-e419a588fa94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:17.428067 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec95ad5-d4a3-451e-9a2e-e419a588fa94 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:17.428374 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec95ad5-d4a3-451e-9a2e-e419a588fa94 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:17.428656 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec95ad5-d4a3-451e-9a2e-e419a588fa94 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:17.439034 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec95ad5-d4a3-451e-9a2e-e419a588fa94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:17.444207 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-fec95ad5-d4a3-451e-9a2e-e419a588fa94 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:17.444207 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:17.444207 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:17.444207 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:17.444207 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:17.444207 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:17.444207 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:17.444207 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:17.444207 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:17.444207 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:17.444207 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:17.444207 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:17.444207 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:17.444207 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:17.444207 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:17.444207 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:17.445126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:17.445126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:17.445126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:17.445126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:17.445126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:17.445126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:17.445126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:17.445126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:17.445126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:17.445126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:17.445126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:17.445126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:17.445126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:17.445126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:17.445126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:17.445126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:17.445900 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:17.445900 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:17.445900 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:17.445900 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:17.445900 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:17.445900 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:17.445900 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:17.445900 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:17.445900 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:17.445900 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:17.445900 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:17.445900 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:17.445900 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 257/501] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 1 16:21:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:17.446572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec95ad5-d4a3-451e-9a2e-e419a588fa94 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:17.516269 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720db6f0-6031-42ba-ac81-2d54dbf75bab None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:17.543094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec95ad5-d4a3-451e-9a2e-e419a588fa94 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:17.548554 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8950a-89b8-4ea4-923f-e13d0dbb30d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:17.549182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8950a-89b8-4ea4-923f-e13d0dbb30d8 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:17.549470 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8950a-89b8-4ea4-923f-e13d0dbb30d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:17.549749 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8950a-89b8-4ea4-923f-e13d0dbb30d8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:17.557331 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce8950a-89b8-4ea4-923f-e13d0dbb30d8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:17.559370 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce8950a-89b8-4ea4-923f-e13d0dbb30d8 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:17.571550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-7ce8950a-89b8-4ea4-923f-e13d0dbb30d8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:21:17.571550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:17.571550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:17.571550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:17.571550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:17.571550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:17.571550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:17.571550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:17.571550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:17.571550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:17.571550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:17.571550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:17.571550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:21:17.571550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:21:17.571550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:21:17.571550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:21:17.572499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:17.572499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:17.572499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:21:17.572499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:17.572499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:21:17.572499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:17.572499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:17.572499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:17.572499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:17.572499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:17.572499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:17.572499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:17.572499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:17.572499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:17.572499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:17.572499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:17.572499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:17.572499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:17.573356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:17.573356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:17.573356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:21:17.573356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:21:17.573356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:21:17.573356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:21:17.573356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:21:17.573356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:21:17.573356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:21:17.573356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:17.573356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 245/502] 10.222.0.18 () {56 vars in 1093 bytes} [Thu Sep 1 16:21:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:17.573898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9dab83-1034-4936-969b-1651e62a812d None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:17.574718 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9dab83-1034-4936-969b-1651e62a812d None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:17.575001 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9dab83-1034-4936-969b-1651e62a812d None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:17.576337 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 246/503] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:17.576961 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9dab83-1034-4936-969b-1651e62a812d None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:17.579797 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88603bf6-ddd5-4a39-9525-7acb65abed32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:17.580404 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88603bf6-ddd5-4a39-9525-7acb65abed32 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:17.580683 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88603bf6-ddd5-4a39-9525-7acb65abed32 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:17.580954 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88603bf6-ddd5-4a39-9525-7acb65abed32 None admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:17.581697 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88603bf6-ddd5-4a39-9525-7acb65abed32 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:17.581999 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88603bf6-ddd5-4a39-9525-7acb65abed32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:17.583496 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88603bf6-ddd5-4a39-9525-7acb65abed32 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:17.619661 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 258/504] 10.222.0.18 () {56 vars in 1137 bytes} [Thu Sep 1 16:21:17 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:17.620400 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88603bf6-ddd5-4a39-9525-7acb65abed32 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:17.644315 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:17.670887 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:17.678900 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9dab83-1034-4936-969b-1651e62a812d None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:17.711879 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0f845-b4b1-49cb-9380-2353c23bf593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:17.712718 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0f845-b4b1-49cb-9380-2353c23bf593 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:17.713007 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0f845-b4b1-49cb-9380-2353c23bf593 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:17.713282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0f845-b4b1-49cb-9380-2353c23bf593 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:17.713918 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a0f845-b4b1-49cb-9380-2353c23bf593 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:17.714224 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a0f845-b4b1-49cb-9380-2353c23bf593 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:17.715650 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a0f845-b4b1-49cb-9380-2353c23bf593 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:17.719802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88603bf6-ddd5-4a39-9525-7acb65abed32 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:17.727017 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 247/505] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:17.727807 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0f845-b4b1-49cb-9380-2353c23bf593 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:17.754747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025be55-70dc-437d-8506-7697a8e0eff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:17.755362 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025be55-70dc-437d-8506-7697a8e0eff8 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:17.755646 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025be55-70dc-437d-8506-7697a8e0eff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:17.755922 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025be55-70dc-437d-8506-7697a8e0eff8 None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:17.756583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d025be55-70dc-437d-8506-7697a8e0eff8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:17.756893 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d025be55-70dc-437d-8506-7697a8e0eff8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:17.758386 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d025be55-70dc-437d-8506-7697a8e0eff8 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:17.766872 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 259/506] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 1 16:21:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:17.768023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f01b9c-d80c-4884-866f-23e3078cdaeb None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:17.768312 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f01b9c-d80c-4884-866f-23e3078cdaeb None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:17.768590 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f01b9c-d80c-4884-866f-23e3078cdaeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:17.785518 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-19f01b9c-d80c-4884-866f-23e3078cdaeb None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:17.786737 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-19f01b9c-d80c-4884-866f-23e3078cdaeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:17.829831 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0f845-b4b1-49cb-9380-2353c23bf593 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:17.852139 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 260/507] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:21:17.852964 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f01b9c-d80c-4884-866f-23e3078cdaeb None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:17.859723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:17.870370 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45bdf0-1f91-476e-818b-bdc66a8fb459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:17.871543 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45bdf0-1f91-476e-818b-bdc66a8fb459 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:17.872074 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45bdf0-1f91-476e-818b-bdc66a8fb459 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:17.872344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45bdf0-1f91-476e-818b-bdc66a8fb459 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:17.873080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da45bdf0-1f91-476e-818b-bdc66a8fb459 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:17.873376 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da45bdf0-1f91-476e-818b-bdc66a8fb459 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:17.874846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da45bdf0-1f91-476e-818b-bdc66a8fb459 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:17.902059 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:17.915401 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 248/508] 10.222.0.18 () {56 vars in 1133 bytes} [Thu Sep 1 16:21:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:17.915401 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45bdf0-1f91-476e-818b-bdc66a8fb459 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:17.964059 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f01b9c-d80c-4884-866f-23e3078cdaeb None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:18.000910 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d6133-3c86-401d-8b15-bdc2e217933c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:18.001524 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d6133-3c86-401d-8b15-bdc2e217933c None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:18.001810 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d6133-3c86-401d-8b15-bdc2e217933c None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:18.002086 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d6133-3c86-401d-8b15-bdc2e217933c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:18.016656 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45bdf0-1f91-476e-818b-bdc66a8fb459 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:18.057292 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55d441-2b24-48ae-b30c-94b4890a9b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:18.058087 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55d441-2b24-48ae-b30c-94b4890a9b2d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:18.058438 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55d441-2b24-48ae-b30c-94b4890a9b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:18.058716 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55d441-2b24-48ae-b30c-94b4890a9b2d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:18.065783 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55d441-2b24-48ae-b30c-94b4890a9b2d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:18.067250 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55d441-2b24-48ae-b30c-94b4890a9b2d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:18.075492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d6133-3c86-401d-8b15-bdc2e217933c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:18.077143 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d6133-3c86-401d-8b15-bdc2e217933c None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:18.078792 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-1d55d441-2b24-48ae-b30c-94b4890a9b2d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:21:18.078792 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:18.078792 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:18.078792 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:18.078792 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:18.078792 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:18.078792 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:18.078792 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:18.078792 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:18.078792 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:18.078792 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:18.078792 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:18.078792 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:21:18.078792 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:21:18.078792 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:21:18.078792 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:21:18.079631 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:18.079631 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:18.079631 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:21:18.079631 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:18.079631 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:21:18.079631 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:18.079631 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:18.079631 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:18.079631 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:18.079631 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:18.079631 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:18.079631 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:18.079631 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:18.079631 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:18.079631 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:18.079631 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:18.079631 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:18.079631 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:18.080494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:18.080494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:18.080494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:21:18.080494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:21:18.080494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:21:18.080494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:21:18.080494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:21:18.080494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:21:18.080494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:21:18.080494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:18.080494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 249/509] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:21:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:18.081033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55d441-2b24-48ae-b30c-94b4890a9b2d None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:18.128860 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:18.129637 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:18.182057 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55d441-2b24-48ae-b30c-94b4890a9b2d None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:18.187082 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 261/510] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:21:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:21:18.187907 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d6133-3c86-401d-8b15-bdc2e217933c None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:18.218451 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d873e8-e53b-4841-affa-3338ed2faa9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:18.219123 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d873e8-e53b-4841-affa-3338ed2faa9b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:18.219409 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d873e8-e53b-4841-affa-3338ed2faa9b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:18.219688 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d873e8-e53b-4841-affa-3338ed2faa9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:18.231196 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d873e8-e53b-4841-affa-3338ed2faa9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:18.232635 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d873e8-e53b-4841-affa-3338ed2faa9b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:18.246938 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-b5d873e8-e53b-4841-affa-3338ed2faa9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:18.246938 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:18.246938 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:18.246938 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:18.246938 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:18.246938 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:18.246938 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:18.246938 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:18.246938 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:18.246938 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:18.246938 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:18.246938 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:18.246938 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:18.246938 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:18.246938 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:18.246938 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:18.247831 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:18.247831 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:18.247831 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:18.247831 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:18.247831 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:18.247831 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:18.247831 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:18.247831 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:18.247831 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:18.247831 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:18.247831 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:18.247831 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:18.247831 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:18.247831 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:18.247831 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:18.247831 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:18.247831 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:18.248890 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:18.248890 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:18.248890 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:18.248890 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 250/511] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:18.248890 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d873e8-e53b-4841-affa-3338ed2faa9b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:18.286692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d6133-3c86-401d-8b15-bdc2e217933c None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:18.320572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aef2a2-b235-4787-af88-fa24fe08207b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:18.321172 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aef2a2-b235-4787-af88-fa24fe08207b None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:18.321451 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aef2a2-b235-4787-af88-fa24fe08207b None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:18.321724 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aef2a2-b235-4787-af88-fa24fe08207b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:18.333148 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:18.345934 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:18.350018 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d873e8-e53b-4841-affa-3338ed2faa9b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:18.384356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b94183-4169-455c-a8e7-b204e07c9938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:18.384900 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b94183-4169-455c-a8e7-b204e07c9938 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:18.385182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b94183-4169-455c-a8e7-b204e07c9938 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:18.385453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b94183-4169-455c-a8e7-b204e07c9938 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:18.386177 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b94183-4169-455c-a8e7-b204e07c9938 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:18.387101 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b94183-4169-455c-a8e7-b204e07c9938 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:18.388551 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b94183-4169-455c-a8e7-b204e07c9938 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:18.390955 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aef2a2-b235-4787-af88-fa24fe08207b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:18.392425 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aef2a2-b235-4787-af88-fa24fe08207b None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:18.405716 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 251/512] 10.222.0.18 () {56 vars in 1137 bytes} [Thu Sep 1 16:21:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:18.406546 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b94183-4169-455c-a8e7-b204e07c9938 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:18.491235 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 262/513] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:21:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:21:18.492069 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aef2a2-b235-4787-af88-fa24fe08207b None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:18.508254 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b94183-4169-455c-a8e7-b204e07c9938 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:18.534466 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:18.542802 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777a6d4-8df5-482a-a191-999da2c07c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:18.543420 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777a6d4-8df5-482a-a191-999da2c07c63 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:18.543701 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777a6d4-8df5-482a-a191-999da2c07c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:18.543971 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777a6d4-8df5-482a-a191-999da2c07c63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:18.553756 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8777a6d4-8df5-482a-a191-999da2c07c63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:18.555211 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8777a6d4-8df5-482a-a191-999da2c07c63 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:18.569303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-8777a6d4-8df5-482a-a191-999da2c07c63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:18.569303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:18.569303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:18.569303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:18.569303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:18.569303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:18.569303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:18.569303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:18.569303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:18.569303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:18.569303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:18.569303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:18.569303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:18.569303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:18.569303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:18.569303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:18.570620 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:18.570620 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:18.570620 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:18.570620 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:18.570620 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:18.570620 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:18.570620 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:18.570620 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:18.570620 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:18.570620 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:18.570620 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:18.570620 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:18.570620 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:18.570620 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:18.570620 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:18.570620 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:18.570620 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:18.571425 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:18.571425 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:18.571425 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:18.571425 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 252/514] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:18.571425 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777a6d4-8df5-482a-a191-999da2c07c63 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:18.582630 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:18.594956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aef2a2-b235-4787-af88-fa24fe08207b None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:18.629041 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48158d-6ab6-4cbc-8f6b-a82dc4cf8877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:18.629680 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48158d-6ab6-4cbc-8f6b-a82dc4cf8877 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:18.629963 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48158d-6ab6-4cbc-8f6b-a82dc4cf8877 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:18.630246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48158d-6ab6-4cbc-8f6b-a82dc4cf8877 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:18.641515 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c48158d-6ab6-4cbc-8f6b-a82dc4cf8877 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:18.643175 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c48158d-6ab6-4cbc-8f6b-a82dc4cf8877 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:18.658395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-0c48158d-6ab6-4cbc-8f6b-a82dc4cf8877 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:18.658395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:18.658395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:18.658395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:18.658395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:18.658395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:18.658395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:18.658395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:18.658395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:18.658395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:18.658395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:18.658395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:18.658395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:18.658395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:18.658395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:18.658395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:18.659296 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:18.659296 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:18.659296 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:18.659296 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:18.659296 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:18.659296 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:18.659296 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:18.659296 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:18.659296 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:18.659296 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:18.659296 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:18.659296 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:18.659296 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:18.659296 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:18.659296 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:18.659296 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:18.659296 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:18.660772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:18.660772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:18.660772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:18.660772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 263/515] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:18.660772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48158d-6ab6-4cbc-8f6b-a82dc4cf8877 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:18.678059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777a6d4-8df5-482a-a191-999da2c07c63 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:18.711564 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9bc54-9ee6-4d9e-aeac-6e0c75b96c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:18.712166 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9bc54-9ee6-4d9e-aeac-6e0c75b96c6e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:18.712446 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9bc54-9ee6-4d9e-aeac-6e0c75b96c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:18.712722 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9bc54-9ee6-4d9e-aeac-6e0c75b96c6e None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:18.713351 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9bc54-9ee6-4d9e-aeac-6e0c75b96c6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:18.713652 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9bc54-9ee6-4d9e-aeac-6e0c75b96c6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:18.715142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9bc54-9ee6-4d9e-aeac-6e0c75b96c6e None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:18.726440 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 253/516] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:18.727219 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9bc54-9ee6-4d9e-aeac-6e0c75b96c6e None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:18.753119 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:18.761059 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48158d-6ab6-4cbc-8f6b-a82dc4cf8877 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:18.794970 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e6fde-0e91-4854-bca7-3ccd10a3ce8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:18.795571 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e6fde-0e91-4854-bca7-3ccd10a3ce8e None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:18.795708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:18.795861 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e6fde-0e91-4854-bca7-3ccd10a3ce8e None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:18.796161 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e6fde-0e91-4854-bca7-3ccd10a3ce8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:18.828029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9bc54-9ee6-4d9e-aeac-6e0c75b96c6e None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:18.861845 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c2093f-db25-4575-baca-c7da247c201a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:18.862503 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2093f-db25-4575-baca-c7da247c201a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:18.862790 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2093f-db25-4575-baca-c7da247c201a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:18.863066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2093f-db25-4575-baca-c7da247c201a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:18.863453 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e6fde-0e91-4854-bca7-3ccd10a3ce8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:18.864921 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e6fde-0e91-4854-bca7-3ccd10a3ce8e None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:18.873274 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c2093f-db25-4575-baca-c7da247c201a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:18.874732 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c2093f-db25-4575-baca-c7da247c201a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:18.888898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-94c2093f-db25-4575-baca-c7da247c201a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:18.888898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:18.888898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:18.888898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:18.888898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:18.888898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:18.888898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:18.888898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:18.888898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:18.888898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:18.888898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:18.888898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:18.888898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:18.888898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:18.888898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:18.888898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:18.889952 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:18.889952 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:18.889952 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:18.889952 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:18.889952 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:18.889952 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:18.889952 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:18.889952 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:18.889952 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:18.889952 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:18.889952 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:18.889952 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:18.889952 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:18.889952 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:18.889952 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:18.889952 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:18.889952 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:18.890854 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:18.890854 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:18.890854 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:18.890854 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 254/517] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:18.891059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c2093f-db25-4575-baca-c7da247c201a None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:18.959484 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:18.967506 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 264/518] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:21:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:21:18.968323 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e6fde-0e91-4854-bca7-3ccd10a3ce8e None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:18.990130 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c2093f-db25-4575-baca-c7da247c201a None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:19.009349 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:19.024109 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f460484-f408-4cdf-964d-20c40e5231ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:19.024717 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f460484-f408-4cdf-964d-20c40e5231ef None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:19.025001 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f460484-f408-4cdf-964d-20c40e5231ef None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:19.025285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f460484-f408-4cdf-964d-20c40e5231ef None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:19.025925 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f460484-f408-4cdf-964d-20c40e5231ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:19.026247 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f460484-f408-4cdf-964d-20c40e5231ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:19.028113 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f460484-f408-4cdf-964d-20c40e5231ef None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:19.039769 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 255/519] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:19.040554 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f460484-f408-4cdf-964d-20c40e5231ef None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:19.065829 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e6fde-0e91-4854-bca7-3ccd10a3ce8e None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:19.099067 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade7ce3b-519e-4848-93d8-5ce6880ff23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:19.099681 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7ce3b-519e-4848-93d8-5ce6880ff23f None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:19.099962 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7ce3b-519e-4848-93d8-5ce6880ff23f None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:19.100235 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7ce3b-519e-4848-93d8-5ce6880ff23f None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:19.100871 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade7ce3b-519e-4848-93d8-5ce6880ff23f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:19.101172 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade7ce3b-519e-4848-93d8-5ce6880ff23f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:19.102580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade7ce3b-519e-4848-93d8-5ce6880ff23f None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:19.119957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 265/520] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:19.120757 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade7ce3b-519e-4848-93d8-5ce6880ff23f None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:19.141053 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f460484-f408-4cdf-964d-20c40e5231ef None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:19.175105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6ed82-3614-4ae5-976f-1a7b59df2693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:19.175731 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6ed82-3614-4ae5-976f-1a7b59df2693 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:19.176022 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6ed82-3614-4ae5-976f-1a7b59df2693 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:19.176295 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6ed82-3614-4ae5-976f-1a7b59df2693 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:19.178720 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:19.183634 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6ed82-3614-4ae5-976f-1a7b59df2693 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:19.185080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6ed82-3614-4ae5-976f-1a7b59df2693 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:19.196561 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-65a6ed82-3614-4ae5-976f-1a7b59df2693 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 16:21:19.196561 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:19.196561 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:19.196561 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:19.196561 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:19.196561 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:19.196561 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:19.196561 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:19.196561 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:19.196561 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:19.196561 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:19.196561 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:19.196561 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:21:19.196561 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:21:19.196561 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:21:19.196561 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:21:19.197406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:19.197406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:19.197406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:21:19.197406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:19.197406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:21:19.197406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:19.197406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:19.197406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:19.197406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:19.197406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:19.197406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:19.197406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:19.197406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:19.197406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:19.197406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:19.197406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:19.197406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:19.197406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:19.198320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:19.198320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:19.198320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:21:19.198320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:21:19.198320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:21:19.198320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:21:19.198320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:21:19.198320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:21:19.198320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 16:21:19.198320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:19.198320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 256/521] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 1 16:21:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:19.198859 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6ed82-3614-4ae5-976f-1a7b59df2693 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:19.225894 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade7ce3b-519e-4848-93d8-5ce6880ff23f None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:19.239822 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:19.259520 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515c574-66fb-4fc2-82b6-4445de5ca713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:19.260126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515c574-66fb-4fc2-82b6-4445de5ca713 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:19.260409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515c574-66fb-4fc2-82b6-4445de5ca713 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:19.260688 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515c574-66fb-4fc2-82b6-4445de5ca713 None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:19.261318 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5515c574-66fb-4fc2-82b6-4445de5ca713 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:19.262133 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5515c574-66fb-4fc2-82b6-4445de5ca713 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:19.263539 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5515c574-66fb-4fc2-82b6-4445de5ca713 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:19.274748 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 266/522] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:19.275384 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515c574-66fb-4fc2-82b6-4445de5ca713 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:19.302303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6ed82-3614-4ae5-976f-1a7b59df2693 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:19.335582 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19020f44-6117-41ba-a6bf-a384e8feecd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:19.336203 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19020f44-6117-41ba-a6bf-a384e8feecd1 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:19.336480 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19020f44-6117-41ba-a6bf-a384e8feecd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:19.336747 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19020f44-6117-41ba-a6bf-a384e8feecd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:19.347853 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19020f44-6117-41ba-a6bf-a384e8feecd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:19.349454 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19020f44-6117-41ba-a6bf-a384e8feecd1 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:19.365502 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-19020f44-6117-41ba-a6bf-a384e8feecd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:19.365502 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:19.365502 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:19.365502 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:19.365502 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:19.365502 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:19.365502 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:19.365502 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:19.365502 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:19.365502 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:19.365502 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:19.365502 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:19.365502 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:19.365502 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:19.365502 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:19.365502 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:19.366432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:19.366432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:19.366432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:19.366432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:19.366432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:19.366432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:19.366432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:19.366432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:19.366432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:19.366432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:19.366432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:19.366432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:19.366432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:19.366432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:19.366432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:19.366432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:19.366432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:19.367328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:19.367328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:19.367328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:19.367328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 257/523] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:19.367328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19020f44-6117-41ba-a6bf-a384e8feecd1 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:19.377641 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515c574-66fb-4fc2-82b6-4445de5ca713 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:19.392411 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:19.413204 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27496cde-677e-4ab8-a9a7-0f19d1fc0458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:19.413810 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27496cde-677e-4ab8-a9a7-0f19d1fc0458 None admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:19.414090 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27496cde-677e-4ab8-a9a7-0f19d1fc0458 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:19.414400 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27496cde-677e-4ab8-a9a7-0f19d1fc0458 None admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:19.415039 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27496cde-677e-4ab8-a9a7-0f19d1fc0458 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:19.416492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27496cde-677e-4ab8-a9a7-0f19d1fc0458 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:19.436829 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 267/524] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 1 16:21:19 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:21:19.437629 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27496cde-677e-4ab8-a9a7-0f19d1fc0458 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:19.458381 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:19.469066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19020f44-6117-41ba-a6bf-a384e8feecd1 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:19.509075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010f02f-f065-4212-8c7f-49736345fda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:19.509636 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010f02f-f065-4212-8c7f-49736345fda8 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:19.509919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010f02f-f065-4212-8c7f-49736345fda8 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:19.510216 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010f02f-f065-4212-8c7f-49736345fda8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:19.518476 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2010f02f-f065-4212-8c7f-49736345fda8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:19.519931 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2010f02f-f065-4212-8c7f-49736345fda8 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:19.532839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-2010f02f-f065-4212-8c7f-49736345fda8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:19.532839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:19.532839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:19.532839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:19.532839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:19.532839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:19.532839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:19.532839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:19.532839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:19.532839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:19.532839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:19.532839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:19.532839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:21:19.532839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:21:19.532839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:21:19.533751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:21:19.533751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:19.533751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:19.533751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:19.533751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:19.533751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:19.533751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:19.533751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:19.533751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:19.533751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:19.533751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:19.533751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:19.533751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:19.533751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:19.533751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:19.533751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:19.533751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:19.534626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:21:19.534626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:21:19.534626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:21:19.534626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:21:19.534626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:21:19.534626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:21:19.534626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:19.534626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:19.534626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 258/525] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:19.535744 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010f02f-f065-4212-8c7f-49736345fda8 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:19.575017 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27496cde-677e-4ab8-a9a7-0f19d1fc0458 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:19.614019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c7be2-1721-4dc9-8e7b-90afd93d78e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:19.614865 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c7be2-1721-4dc9-8e7b-90afd93d78e6 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:19.615145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c7be2-1721-4dc9-8e7b-90afd93d78e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:19.615737 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c7be2-1721-4dc9-8e7b-90afd93d78e6 None admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:19.616738 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c7be2-1721-4dc9-8e7b-90afd93d78e6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:19.617048 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c7be2-1721-4dc9-8e7b-90afd93d78e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:19.618514 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c7be2-1721-4dc9-8e7b-90afd93d78e6 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:19.642059 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 268/526] 10.222.0.18 () {56 vars in 1139 bytes} [Thu Sep 1 16:21:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:19.642938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c7be2-1721-4dc9-8e7b-90afd93d78e6 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:19.658449 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010f02f-f065-4212-8c7f-49736345fda8 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:19.678783 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:19.694133 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96581527-26c5-43ab-b1aa-afeba878195a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:19.694791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96581527-26c5-43ab-b1aa-afeba878195a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:19.695070 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96581527-26c5-43ab-b1aa-afeba878195a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:19.695338 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96581527-26c5-43ab-b1aa-afeba878195a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:19.703219 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96581527-26c5-43ab-b1aa-afeba878195a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:19.704706 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96581527-26c5-43ab-b1aa-afeba878195a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:19.719660 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-96581527-26c5-43ab-b1aa-afeba878195a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:19.719660 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:19.719660 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:19.719660 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:19.719660 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:19.719660 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:19.719660 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:19.719660 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:19.719660 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:19.719660 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:19.719660 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:19.719660 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:19.719660 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:21:19.719660 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:21:19.719660 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:21:19.720534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:21:19.720534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:19.720534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:19.720534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:19.720534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:19.720534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:19.720534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:19.720534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:19.720534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:19.720534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:19.720534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:19.720534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:19.720534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:19.720534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:19.720534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:19.720534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:19.720534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:19.721472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:21:19.721472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:21:19.721472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:21:19.721472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:21:19.721472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:21:19.721472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:21:19.721472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:19.721472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:19.721472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 259/527] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:19.722742 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96581527-26c5-43ab-b1aa-afeba878195a None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:19.738803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:19.756428 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c7be2-1721-4dc9-8e7b-90afd93d78e6 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:19.796400 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65262d1f-d0bc-40d8-8525-0190c4d3d842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:19.797704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65262d1f-d0bc-40d8-8525-0190c4d3d842 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:19.797704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65262d1f-d0bc-40d8-8525-0190c4d3d842 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:19.797704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65262d1f-d0bc-40d8-8525-0190c4d3d842 None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:19.798292 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65262d1f-d0bc-40d8-8525-0190c4d3d842 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:19.798599 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65262d1f-d0bc-40d8-8525-0190c4d3d842 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:19.799965 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65262d1f-d0bc-40d8-8525-0190c4d3d842 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:19.817060 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 269/528] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:19.817060 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65262d1f-d0bc-40d8-8525-0190c4d3d842 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:19.833457 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96581527-26c5-43ab-b1aa-afeba878195a None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:19.870293 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1cf28e-829b-4ad3-82d6-1f1628aef587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:19.871830 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1cf28e-829b-4ad3-82d6-1f1628aef587 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:19.872116 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1cf28e-829b-4ad3-82d6-1f1628aef587 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:19.872390 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1cf28e-829b-4ad3-82d6-1f1628aef587 None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:19.873087 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1cf28e-829b-4ad3-82d6-1f1628aef587 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:19.873700 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1cf28e-829b-4ad3-82d6-1f1628aef587 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:19.875000 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1cf28e-829b-4ad3-82d6-1f1628aef587 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:19.898320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 260/529] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:21:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:19.898320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1cf28e-829b-4ad3-82d6-1f1628aef587 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:19.910373 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:19.923831 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65262d1f-d0bc-40d8-8525-0190c4d3d842 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:19.960402 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83704553-b34a-41c1-b64a-efcc035f7807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:19.961165 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83704553-b34a-41c1-b64a-efcc035f7807 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:19.961451 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83704553-b34a-41c1-b64a-efcc035f7807 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:19.961725 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83704553-b34a-41c1-b64a-efcc035f7807 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:19.976323 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:20.009632 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1cf28e-829b-4ad3-82d6-1f1628aef587 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:20.033801 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83704553-b34a-41c1-b64a-efcc035f7807 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:20.035302 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83704553-b34a-41c1-b64a-efcc035f7807 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:20.044926 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c3b67-fcc3-434b-a7e5-8a2e94dd5202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:20.045528 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c3b67-fcc3-434b-a7e5-8a2e94dd5202 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:20.045799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c3b67-fcc3-434b-a7e5-8a2e94dd5202 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:20.046206 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c3b67-fcc3-434b-a7e5-8a2e94dd5202 None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:20.046970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38c3b67-fcc3-434b-a7e5-8a2e94dd5202 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:20.047220 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38c3b67-fcc3-434b-a7e5-8a2e94dd5202 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:20.048635 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38c3b67-fcc3-434b-a7e5-8a2e94dd5202 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:20.066350 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 261/530] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:21:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:20.067491 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c3b67-fcc3-434b-a7e5-8a2e94dd5202 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:20.128303 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:20.141499 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 270/531] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:21:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:21:20.142508 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83704553-b34a-41c1-b64a-efcc035f7807 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:20.171876 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c3b67-fcc3-434b-a7e5-8a2e94dd5202 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:20.202126 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:20.208584 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576338dd-29f0-4c91-b8dc-d231215d079e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:20.209472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576338dd-29f0-4c91-b8dc-d231215d079e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:20.210229 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576338dd-29f0-4c91-b8dc-d231215d079e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:20.210890 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576338dd-29f0-4c91-b8dc-d231215d079e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:20.217399 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576338dd-29f0-4c91-b8dc-d231215d079e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:20.219052 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576338dd-29f0-4c91-b8dc-d231215d079e None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:20.230050 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-576338dd-29f0-4c91-b8dc-d231215d079e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:20.230050 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:20.230050 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:20.230050 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:20.230050 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:20.230050 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:20.230050 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:20.230050 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:20.230050 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:20.230050 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:20.230050 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:20.230050 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:20.230050 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:21:20.230050 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:21:20.230050 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:21:20.231453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:21:20.231453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:20.231453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:20.231453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:20.231453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:20.231453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:20.231453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:20.231453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:20.231453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:20.231453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:20.231453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:20.231453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:20.231453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:20.231453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:20.231453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:20.231453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:20.231453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:20.232268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:21:20.232268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:21:20.232268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:21:20.232268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:21:20.232268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:21:20.232268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:21:20.232268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:20.232268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:20.232268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 262/532] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:20.233238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576338dd-29f0-4c91-b8dc-d231215d079e None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:20.250386 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83704553-b34a-41c1-b64a-efcc035f7807 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:20.286128 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658aa8a7-1a26-4646-bfba-05c0ec0cdf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:20.286780 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658aa8a7-1a26-4646-bfba-05c0ec0cdf88 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:20.287061 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658aa8a7-1a26-4646-bfba-05c0ec0cdf88 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:20.287335 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658aa8a7-1a26-4646-bfba-05c0ec0cdf88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:20.288062 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658aa8a7-1a26-4646-bfba-05c0ec0cdf88 None admin] RBAC: Adding query filter params (role.id=b6727ed962fa41109b0a2d6851551f49, scope.project.id=36ce80c4e0d64562a9ab3961be76fad9, user.id=b09f6d550c9142bf956b225bf5172e42) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:20.288373 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658aa8a7-1a26-4646-bfba-05c0ec0cdf88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:20.289810 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658aa8a7-1a26-4646-bfba-05c0ec0cdf88 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:20.295881 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 271/533] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:21:20 2022] GET /identity/v3/role_assignments?user.id=b09f6d550c9142bf956b225bf5172e42&scope.project.id=36ce80c4e0d64562a9ab3961be76fad9&role.id=b6727ed962fa41109b0a2d6851551f49 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:20.299731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658aa8a7-1a26-4646-bfba-05c0ec0cdf88 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:20.356318 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576338dd-29f0-4c91-b8dc-d231215d079e None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:20.379167 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:20.415437 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b0e38-a32a-47b8-9796-52ee6caa3800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:20.416392 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b0e38-a32a-47b8-9796-52ee6caa3800 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:20.417275 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b0e38-a32a-47b8-9796-52ee6caa3800 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:20.418362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b0e38-a32a-47b8-9796-52ee6caa3800 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:20.419570 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b0e38-a32a-47b8-9796-52ee6caa3800 None admin] RBAC: Adding query filter params (role.id=b6727ed962fa41109b0a2d6851551f49, scope.project.id=36ce80c4e0d64562a9ab3961be76fad9, user.id=278e6d6aed414c758fe3e9f4df6bf81e) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:20.420089 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b0e38-a32a-47b8-9796-52ee6caa3800 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:20.422819 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b0e38-a32a-47b8-9796-52ee6caa3800 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:20.436433 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 263/534] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:21:20 2022] GET /identity/v3/role_assignments?user.id=278e6d6aed414c758fe3e9f4df6bf81e&scope.project.id=36ce80c4e0d64562a9ab3961be76fad9&role.id=b6727ed962fa41109b0a2d6851551f49 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:20.436433 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b0e38-a32a-47b8-9796-52ee6caa3800 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:20.456902 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658aa8a7-1a26-4646-bfba-05c0ec0cdf88 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:20.506571 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8ce34-2517-4e1d-afa9-fe34769ebc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:20.507529 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8ce34-2517-4e1d-afa9-fe34769ebc24 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:20.507815 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8ce34-2517-4e1d-afa9-fe34769ebc24 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:20.508092 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8ce34-2517-4e1d-afa9-fe34769ebc24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:20.509850 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:20.523975 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a8ce34-2517-4e1d-afa9-fe34769ebc24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:20.525491 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a8ce34-2517-4e1d-afa9-fe34769ebc24 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:20.544895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-d7a8ce34-2517-4e1d-afa9-fe34769ebc24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:20.544895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:20.544895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:20.544895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:20.544895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:20.544895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:20.544895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:20.544895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:20.544895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:20.544895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:20.544895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:20.544895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:20.544895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:20.544895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:20.544895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:20.544895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:20.545855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:20.545855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:20.545855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:20.545855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:20.545855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:20.545855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:20.545855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:20.545855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:20.545855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:20.545855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:20.545855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:20.545855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:20.545855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:20.545855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:20.545855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:20.545855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:20.545855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:20.546695 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:20.546695 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:20.546695 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:20.546695 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 272/535] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:20.560874 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8ce34-2517-4e1d-afa9-fe34769ebc24 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:20.615679 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b0e38-a32a-47b8-9796-52ee6caa3800 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:20.653755 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704f863-0c60-4004-ab7e-e2cb2f6f7967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:20.654407 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704f863-0c60-4004-ab7e-e2cb2f6f7967 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:20.654962 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704f863-0c60-4004-ab7e-e2cb2f6f7967 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:20.655336 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704f863-0c60-4004-ab7e-e2cb2f6f7967 None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:20.656136 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c704f863-0c60-4004-ab7e-e2cb2f6f7967 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:20.656903 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c704f863-0c60-4004-ab7e-e2cb2f6f7967 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:20.658460 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c704f863-0c60-4004-ab7e-e2cb2f6f7967 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:20.671526 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 264/536] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:21:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:20.680564 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704f863-0c60-4004-ab7e-e2cb2f6f7967 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:20.726157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8ce34-2517-4e1d-afa9-fe34769ebc24 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:20.729038 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:20.764608 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4244c-2c70-4112-ad05-c15cd5e08fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:20.766137 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4244c-2c70-4112-ad05-c15cd5e08fcd None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:20.766647 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4244c-2c70-4112-ad05-c15cd5e08fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:20.767119 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4244c-2c70-4112-ad05-c15cd5e08fcd None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:20.768066 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d4244c-2c70-4112-ad05-c15cd5e08fcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:20.768568 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d4244c-2c70-4112-ad05-c15cd5e08fcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:20.770292 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d4244c-2c70-4112-ad05-c15cd5e08fcd None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:20.783333 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 273/537] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:20.791302 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4244c-2c70-4112-ad05-c15cd5e08fcd None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:20.798354 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704f863-0c60-4004-ab7e-e2cb2f6f7967 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:20.844353 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:20.863382 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c210d-33e7-4fd5-8d8f-2efc6611e9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:20.864094 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c210d-33e7-4fd5-8d8f-2efc6611e9ba None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:20.864285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c210d-33e7-4fd5-8d8f-2efc6611e9ba None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:20.864563 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c210d-33e7-4fd5-8d8f-2efc6611e9ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:20.865483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7c210d-33e7-4fd5-8d8f-2efc6611e9ba None admin] RBAC: Adding query filter params (role.id=b6727ed962fa41109b0a2d6851551f49, scope.project.id=36ce80c4e0d64562a9ab3961be76fad9, user.id=b78747bb9140457896fe62d4f537592c) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:20.865638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7c210d-33e7-4fd5-8d8f-2efc6611e9ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:20.867152 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7c210d-33e7-4fd5-8d8f-2efc6611e9ba None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:20.873564 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 265/538] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:21:20 2022] GET /identity/v3/role_assignments?user.id=b78747bb9140457896fe62d4f537592c&scope.project.id=36ce80c4e0d64562a9ab3961be76fad9&role.id=b6727ed962fa41109b0a2d6851551f49 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:20.913804 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4244c-2c70-4112-ad05-c15cd5e08fcd None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:20.983523 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3c288-f35e-4f96-8ca6-e79abc2dfff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:20.984536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3c288-f35e-4f96-8ca6-e79abc2dfff9 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:20.984967 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3c288-f35e-4f96-8ca6-e79abc2dfff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:20.985387 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3c288-f35e-4f96-8ca6-e79abc2dfff9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:20.997413 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec3c288-f35e-4f96-8ca6-e79abc2dfff9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:21.004034 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec3c288-f35e-4f96-8ca6-e79abc2dfff9 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:21.018633 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-3ec3c288-f35e-4f96-8ca6-e79abc2dfff9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:21.018633 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:21.018633 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:21.018633 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:21.018633 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:21.018633 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:21.018633 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:21.018633 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:21.018633 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:21.018633 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:21.018633 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:21.018633 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:21.018633 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:21:21.018633 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:21:21.018633 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:21:21.019519 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:21:21.019519 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:21.019519 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:21.019519 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:21.019519 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:21.019519 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:21.019519 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:21.019519 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:21.019519 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:21.019519 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:21.019519 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:21.019519 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:21.019519 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:21.019519 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:21.019519 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:21.019519 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:21.019519 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:21.020329 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:21:21.020329 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:21:21.020329 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:21:21.020329 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:21:21.020329 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:21:21.020329 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:21:21.020329 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:21.020329 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:21.022732 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 274/539] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:21.028501 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c210d-33e7-4fd5-8d8f-2efc6611e9ba None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:21.174827 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c210d-33e7-4fd5-8d8f-2efc6611e9ba None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:21.228918 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06944241-0fa1-464d-973f-b8763816e685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:21.229541 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06944241-0fa1-464d-973f-b8763816e685 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:21.229828 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06944241-0fa1-464d-973f-b8763816e685 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:21.230140 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06944241-0fa1-464d-973f-b8763816e685 None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:21.230921 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06944241-0fa1-464d-973f-b8763816e685 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:21.231230 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06944241-0fa1-464d-973f-b8763816e685 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:21.232732 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06944241-0fa1-464d-973f-b8763816e685 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:21.251936 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 266/540] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:21:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:21.257609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3c288-f35e-4f96-8ca6-e79abc2dfff9 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:21.290599 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:21.371665 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3c288-f35e-4f96-8ca6-e79abc2dfff9 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:21.419681 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36158f-84bf-493d-ae54-fb366da17529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:21.420685 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36158f-84bf-493d-ae54-fb366da17529 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:21.421170 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36158f-84bf-493d-ae54-fb366da17529 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:21.421643 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36158f-84bf-493d-ae54-fb366da17529 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:21.422599 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db36158f-84bf-493d-ae54-fb366da17529 None admin] RBAC: Adding query filter params (role.id=b6727ed962fa41109b0a2d6851551f49, scope.project.id=36ce80c4e0d64562a9ab3961be76fad9, user.id=ffa0f2160a934906b59e6f845edb0776) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:21.423144 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db36158f-84bf-493d-ae54-fb366da17529 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:21.424791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db36158f-84bf-493d-ae54-fb366da17529 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:21.431245 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 275/541] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:21:21 2022] GET /identity/v3/role_assignments?user.id=ffa0f2160a934906b59e6f845edb0776&scope.project.id=36ce80c4e0d64562a9ab3961be76fad9&role.id=b6727ed962fa41109b0a2d6851551f49 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:21.550442 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb44420-882a-43ff-a04f-eaffd1581627 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:21.551080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb44420-882a-43ff-a04f-eaffd1581627 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:21.551598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb44420-882a-43ff-a04f-eaffd1581627 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:21.553459 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 267/542] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:21.560115 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc45df7-ec83-4c5d-a96b-648bf2a79f44 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:21.560735 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc45df7-ec83-4c5d-a96b-648bf2a79f44 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:21.561246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc45df7-ec83-4c5d-a96b-648bf2a79f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:21.591414 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-cbc45df7-ec83-4c5d-a96b-648bf2a79f44 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:21.592663 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-cbc45df7-ec83-4c5d-a96b-648bf2a79f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:21.695445 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 276/543] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:21:22.073456 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb44420-882a-43ff-a04f-eaffd1581627 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:22.122433 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:22.200370 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb44420-882a-43ff-a04f-eaffd1581627 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:22.251236 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1192df9-62d6-45e9-acc1-81349281f55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:22.251867 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1192df9-62d6-45e9-acc1-81349281f55f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:22.252150 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1192df9-62d6-45e9-acc1-81349281f55f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:22.252424 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1192df9-62d6-45e9-acc1-81349281f55f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:22.269929 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1192df9-62d6-45e9-acc1-81349281f55f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:22.274722 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-f1192df9-62d6-45e9-acc1-81349281f55f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:22.274722 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:22.274722 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:22.274722 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:22.274722 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:22.274722 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:22.274722 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:22.274722 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:22.274722 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:22.274722 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:22.274722 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:22.274722 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:22.274722 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:22.274722 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:22.274722 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:22.274722 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:22.275550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:22.275550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:22.275550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:22.275550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:22.275550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:22.275550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:22.275550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:22.275550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:22.275550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:22.275550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:22.275550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:22.275550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:22.275550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:22.275550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:22.275550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:22.275550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:22.276341 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:22.276341 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:22.276341 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:22.276341 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:22.276341 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:22.276341 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:22.276341 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:22.276341 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:22.276341 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:22.276341 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:22.276341 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:22.276341 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:22.276341 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 268/544] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 1 16:21:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:22.282446 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc45df7-ec83-4c5d-a96b-648bf2a79f44 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:22.312767 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:22.427826 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc45df7-ec83-4c5d-a96b-648bf2a79f44 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:22.463397 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea94bb-a451-4c22-a336-39d6ba1eceab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:22.464013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea94bb-a451-4c22-a336-39d6ba1eceab None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:22.464296 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea94bb-a451-4c22-a336-39d6ba1eceab None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:22.465132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea94bb-a451-4c22-a336-39d6ba1eceab None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:22.465805 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea94bb-a451-4c22-a336-39d6ba1eceab None admin] RBAC: Adding query filter params (name=service) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:22.466114 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea94bb-a451-4c22-a336-39d6ba1eceab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:22.467556 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea94bb-a451-4c22-a336-39d6ba1eceab None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:22.476206 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 277/545] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 1 16:21:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:22.481845 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1192df9-62d6-45e9-acc1-81349281f55f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:22.631397 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:22.634797 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1192df9-62d6-45e9-acc1-81349281f55f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:22.681720 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f832942-f213-4a50-9542-1e27d58463d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:22.682446 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f832942-f213-4a50-9542-1e27d58463d2 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:22.682726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f832942-f213-4a50-9542-1e27d58463d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:22.682997 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f832942-f213-4a50-9542-1e27d58463d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:22.699685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f832942-f213-4a50-9542-1e27d58463d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:22.701173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f832942-f213-4a50-9542-1e27d58463d2 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:22.719912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-0f832942-f213-4a50-9542-1e27d58463d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:22.719912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:22.719912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:22.719912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:22.719912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:22.719912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:22.719912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:22.719912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:22.719912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:22.719912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:22.719912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:22.719912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:22.719912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:22.719912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:22.719912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:22.719912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:22.720871 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:22.720871 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:22.720871 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:22.720871 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:22.720871 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:22.720871 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:22.720871 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:22.720871 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:22.720871 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:22.720871 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:22.720871 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:22.720871 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:22.720871 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:22.720871 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:22.720871 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:22.720871 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:22.720871 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:22.721671 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:22.721671 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:22.721671 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:22.721671 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 269/546] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:22.728209 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea94bb-a451-4c22-a336-39d6ba1eceab None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:22.854267 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:22.869907 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea94bb-a451-4c22-a336-39d6ba1eceab None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:22.907873 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5315b-0ee4-4031-a940-0fc9e8267629 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:22.908128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5315b-0ee4-4031-a940-0fc9e8267629 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:22.908403 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5315b-0ee4-4031-a940-0fc9e8267629 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:22.909986 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 270/547] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:21:22.920814 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d16e2d-df00-4b6d-9ff8-ecf0f842155c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:22.921435 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d16e2d-df00-4b6d-9ff8-ecf0f842155c None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:22.921720 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d16e2d-df00-4b6d-9ff8-ecf0f842155c None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:22.921996 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d16e2d-df00-4b6d-9ff8-ecf0f842155c None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:22.922676 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d16e2d-df00-4b6d-9ff8-ecf0f842155c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:22.922988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d16e2d-df00-4b6d-9ff8-ecf0f842155c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:22.924362 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d16e2d-df00-4b6d-9ff8-ecf0f842155c None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:22.928434 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a194726-4c9b-4b54-af2c-5da68472b462 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:22.928718 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a194726-4c9b-4b54-af2c-5da68472b462 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:22.928996 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a194726-4c9b-4b54-af2c-5da68472b462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:22.938628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 278/548] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:22.943955 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d16e2d-df00-4b6d-9ff8-ecf0f842155c None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:22.975597 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-8a194726-4c9b-4b54-af2c-5da68472b462 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:22.978418 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8a194726-4c9b-4b54-af2c-5da68472b462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:23.084302 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 271/549] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:21:23.086389 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904baa8f-eedd-4cc6-8900-58c0f5c79c07 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:23.086875 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904baa8f-eedd-4cc6-8900-58c0f5c79c07 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:23.087403 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904baa8f-eedd-4cc6-8900-58c0f5c79c07 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:23.088835 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 272/550] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:23.094751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c870a-71be-4c68-a0ab-b4b0b4c4752e None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:23.095038 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c870a-71be-4c68-a0ab-b4b0b4c4752e None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:23.095314 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c870a-71be-4c68-a0ab-b4b0b4c4752e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:23.096292 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d16e2d-df00-4b6d-9ff8-ecf0f842155c None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:23.122925 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-1b9c870a-71be-4c68-a0ab-b4b0b4c4752e None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:23.124137 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-1b9c870a-71be-4c68-a0ab-b4b0b4c4752e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:23.143836 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da052880-8240-4e9a-af13-d5f87a6a34bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:23.144521 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da052880-8240-4e9a-af13-d5f87a6a34bb None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:23.144817 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da052880-8240-4e9a-af13-d5f87a6a34bb None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:23.145101 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da052880-8240-4e9a-af13-d5f87a6a34bb None admin] PATH_INFO: `/v3/users/swift` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:23.153101 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da052880-8240-4e9a-af13-d5f87a6a34bb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:23.154581 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da052880-8240-4e9a-af13-d5f87a6a34bb None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:23.157755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:23.168310 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-da052880-8240-4e9a-af13-d5f87a6a34bb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:21:23.168310 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:23.168310 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:23.168310 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:23.168310 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:23.168310 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:23.168310 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:23.168310 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:23.168310 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:23.168310 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:23.168310 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:23.168310 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:23.168310 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:21:23.168310 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:21:23.168310 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:21:23.168310 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:21:23.169418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:23.169418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:23.169418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:21:23.169418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:23.169418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:21:23.169418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:23.169418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:23.169418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:23.169418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:23.169418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:23.169418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:23.169418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:23.169418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:23.169418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:23.169418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:23.169418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:23.169418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:23.169418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:23.170334 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:23.170334 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:23.170334 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:21:23.170334 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:21:23.170334 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:21:23.170334 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:21:23.170334 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:21:23.170334 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:21:23.170334 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:21:23.170334 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:23.174158 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 279/551] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:21:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:23.175703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd2b72-ca12-4e01-b11e-52e6d72686e3 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:23.175993 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd2b72-ca12-4e01-b11e-52e6d72686e3 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:23.176274 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd2b72-ca12-4e01-b11e-52e6d72686e3 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:23.177787 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 280/552] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:23.182481 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd2b72-ca12-4e01-b11e-52e6d72686e3 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:23.219363 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 273/553] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:21:23.222007 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58441954-da65-44df-8821-beedfd879d6c None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:23.222547 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58441954-da65-44df-8821-beedfd879d6c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:23.223022 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58441954-da65-44df-8821-beedfd879d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:23.246999 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-58441954-da65-44df-8821-beedfd879d6c None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:23.248515 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-58441954-da65-44df-8821-beedfd879d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:23.260307 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:23.322871 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 274/554] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:21:23.354987 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd2b72-ca12-4e01-b11e-52e6d72686e3 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:23.409675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d5a39-562f-420f-a438-924bafa54dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:23.410324 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d5a39-562f-420f-a438-924bafa54dce None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:23.410613 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d5a39-562f-420f-a438-924bafa54dce None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:23.410893 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d5a39-562f-420f-a438-924bafa54dce None admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:23.411625 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d5a39-562f-420f-a438-924bafa54dce None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:23.411936 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d5a39-562f-420f-a438-924bafa54dce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:23.413375 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d5a39-562f-420f-a438-924bafa54dce None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:23.417172 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58441954-da65-44df-8821-beedfd879d6c None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:23.443851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 281/555] 10.222.0.18 () {56 vars in 1135 bytes} [Thu Sep 1 16:21:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:23.451065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d5a39-562f-420f-a438-924bafa54dce None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:23.536331 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:23.577524 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58441954-da65-44df-8821-beedfd879d6c None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:23.585938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d5a39-562f-420f-a438-924bafa54dce None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:23.618847 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4104fba7-d3ec-40f0-b7cf-6a637901f652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:23.619594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4104fba7-d3ec-40f0-b7cf-6a637901f652 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:23.619891 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4104fba7-d3ec-40f0-b7cf-6a637901f652 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:23.620170 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4104fba7-d3ec-40f0-b7cf-6a637901f652 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:23.624264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b4b73-1b6b-471f-8800-c05e880c85ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:23.625034 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b4b73-1b6b-471f-8800-c05e880c85ad None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:23.625456 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b4b73-1b6b-471f-8800-c05e880c85ad None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:23.625867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b4b73-1b6b-471f-8800-c05e880c85ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:23.633327 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4104fba7-d3ec-40f0-b7cf-6a637901f652 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:23.638677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-4104fba7-d3ec-40f0-b7cf-6a637901f652 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:23.638677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:23.638677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:23.638677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:23.638677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:23.638677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:23.638677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:23.638677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:23.638677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:23.638677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:23.638677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:23.638677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:23.638677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:23.638677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:23.638677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:23.638677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:23.639538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:23.639538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:23.639538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:23.639538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:23.639538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:23.639538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:23.639538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:23.639538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:23.639538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:23.639538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:23.639538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:23.639538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:23.639538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:23.639538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:23.639538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:23.639538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:23.640328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:23.640328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:23.640328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:23.640328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:23.640328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:23.640328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:23.640328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:23.640328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:23.640328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:23.640328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:23.640328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:23.640328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:23.640328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 275/556] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 1 16:21:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:23.641015 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4104fba7-d3ec-40f0-b7cf-6a637901f652 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:23.643745 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:23.706215 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6b4b73-1b6b-471f-8800-c05e880c85ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:23.707639 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6b4b73-1b6b-471f-8800-c05e880c85ad None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:23.743825 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4104fba7-d3ec-40f0-b7cf-6a637901f652 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:23.764430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:23.777368 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5119056d-b7d2-4437-8efd-8210b4803fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:23.777997 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119056d-b7d2-4437-8efd-8210b4803fbc None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:23.778305 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119056d-b7d2-4437-8efd-8210b4803fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:23.778582 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119056d-b7d2-4437-8efd-8210b4803fbc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:23.788452 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5119056d-b7d2-4437-8efd-8210b4803fbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:23.793661 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-5119056d-b7d2-4437-8efd-8210b4803fbc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:23.793661 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:23.793661 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:23.793661 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:23.793661 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:23.793661 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:23.793661 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:23.793661 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:23.793661 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:23.793661 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:23.793661 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:23.793661 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:23.793661 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:23.793661 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:23.793661 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:23.793661 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:23.794742 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:23.794742 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:23.794742 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:23.794742 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:23.794742 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:23.794742 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:23.794742 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:23.794742 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:23.794742 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:23.794742 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:23.794742 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:23.794742 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:23.794742 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:23.794742 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:23.794742 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:23.794742 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:23.795514 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:23.795514 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:23.795514 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:23.795514 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:23.795514 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:23.795514 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:23.795514 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:23.795514 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:23.795514 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:23.795514 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:23.795514 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:23.795514 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:23.795514 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 276/557] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 1 16:21:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:23.796173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138483b-26c8-4cb9-a862-48a2a531012f None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:23.796173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138483b-26c8-4cb9-a862-48a2a531012f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:23.796398 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138483b-26c8-4cb9-a862-48a2a531012f None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:23.801334 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 277/558] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:23.801334 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138483b-26c8-4cb9-a862-48a2a531012f None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:23.812268 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 282/559] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:21:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:21:23.813099 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b4b73-1b6b-471f-8800-c05e880c85ad None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:23.863282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:23.903343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138483b-26c8-4cb9-a862-48a2a531012f None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:23.909403 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b4b73-1b6b-471f-8800-c05e880c85ad None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:23.936758 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69118921-d980-452b-bf94-0a3b9c795bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:23.937360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69118921-d980-452b-bf94-0a3b9c795bea None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:23.937636 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69118921-d980-452b-bf94-0a3b9c795bea None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:23.937904 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69118921-d980-452b-bf94-0a3b9c795bea None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:23.938587 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69118921-d980-452b-bf94-0a3b9c795bea None admin] RBAC: Adding query filter params (name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:23.938886 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69118921-d980-452b-bf94-0a3b9c795bea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:23.940237 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69118921-d980-452b-bf94-0a3b9c795bea None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:23.943065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d1241-e8fd-4420-99b0-84106e1c90f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:23.943687 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d1241-e8fd-4420-99b0-84106e1c90f3 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:23.943968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d1241-e8fd-4420-99b0-84106e1c90f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:23.944239 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d1241-e8fd-4420-99b0-84106e1c90f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:23.947913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 278/560] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 1 16:21:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:23.948713 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69118921-d980-452b-bf94-0a3b9c795bea None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:23.954963 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d1241-e8fd-4420-99b0-84106e1c90f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:23.959802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-f54d1241-e8fd-4420-99b0-84106e1c90f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:23.959802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:23.959802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:23.959802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:23.959802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:23.959802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:23.959802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:23.959802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:23.959802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:23.959802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:23.959802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:23.959802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:23.959802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:23.959802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:23.959802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:23.959802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:23.960770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:23.960770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:23.960770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:23.960770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:23.960770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:23.960770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:23.960770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:23.960770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:23.960770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:23.960770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:23.960770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:23.960770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:23.960770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:23.960770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:23.960770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:23.960770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:23.961693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:23.961693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:23.961693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:23.961693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:23.961693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:23.961693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:23.961693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:23.961693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:23.961693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:23.961693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:23.961693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:23.961693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:23.965446 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 283/561] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 1 16:21:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:23.966640 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4bbba4-67d2-46cd-8bda-7057793b8e3f None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:23.966929 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4bbba4-67d2-46cd-8bda-7057793b8e3f None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:23.967205 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4bbba4-67d2-46cd-8bda-7057793b8e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:23.984619 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-6d4bbba4-67d2-46cd-8bda-7057793b8e3f None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:23.985812 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-6d4bbba4-67d2-46cd-8bda-7057793b8e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:23.999674 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:24.048082 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69118921-d980-452b-bf94-0a3b9c795bea None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:24.052105 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 284/562] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:21:24.052900 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4bbba4-67d2-46cd-8bda-7057793b8e3f None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:24.080260 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:24.092069 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c7165-0b78-4ec3-b185-3e2a04f8e87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:24.092734 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c7165-0b78-4ec3-b185-3e2a04f8e87a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:24.092958 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c7165-0b78-4ec3-b185-3e2a04f8e87a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:24.093230 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c7165-0b78-4ec3-b185-3e2a04f8e87a None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:24.093887 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c7165-0b78-4ec3-b185-3e2a04f8e87a None admin] RBAC: Adding query filter params (name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:24.094190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c7165-0b78-4ec3-b185-3e2a04f8e87a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:24.095567 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c7165-0b78-4ec3-b185-3e2a04f8e87a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:24.104240 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 279/563] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 1 16:21:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:24.105786 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c7165-0b78-4ec3-b185-3e2a04f8e87a None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:24.164714 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4bbba4-67d2-46cd-8bda-7057793b8e3f None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:24.199800 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac62660-06e2-4f1f-a177-d354a3fb28c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:24.200427 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac62660-06e2-4f1f-a177-d354a3fb28c9 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:24.200710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac62660-06e2-4f1f-a177-d354a3fb28c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:24.200983 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac62660-06e2-4f1f-a177-d354a3fb28c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:24.211100 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac62660-06e2-4f1f-a177-d354a3fb28c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:24.212444 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c7165-0b78-4ec3-b185-3e2a04f8e87a None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:24.212566 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac62660-06e2-4f1f-a177-d354a3fb28c9 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:24.222665 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:24.230799 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-1ac62660-06e2-4f1f-a177-d354a3fb28c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:24.230799 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:24.230799 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:24.230799 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:24.230799 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:24.230799 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:24.230799 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:24.230799 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:24.230799 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:24.230799 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:24.230799 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:24.230799 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:24.230799 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:24.230799 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:24.230799 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:24.230799 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:24.231881 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:24.231881 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:24.231881 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:24.231881 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:24.231881 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:24.231881 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:24.231881 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:24.231881 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:24.231881 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:24.231881 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:24.231881 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:24.231881 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:24.231881 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:24.231881 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:24.231881 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:24.231881 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:24.231881 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:24.233013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:24.233013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:24.233013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:24.239188 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 285/564] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:24.239188 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac62660-06e2-4f1f-a177-d354a3fb28c9 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:24.256113 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d100f6a-adf1-4412-9318-7d9ba7501595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:24.256766 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d100f6a-adf1-4412-9318-7d9ba7501595 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:24.257037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d100f6a-adf1-4412-9318-7d9ba7501595 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:24.257494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d100f6a-adf1-4412-9318-7d9ba7501595 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:24.268172 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d100f6a-adf1-4412-9318-7d9ba7501595 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:24.269778 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d100f6a-adf1-4412-9318-7d9ba7501595 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:24.284291 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-1d100f6a-adf1-4412-9318-7d9ba7501595 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:24.284291 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:24.284291 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:24.284291 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:24.284291 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:24.284291 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:24.284291 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:24.284291 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:24.284291 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:24.284291 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:24.284291 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:24.284291 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:24.284291 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:24.284291 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:24.284291 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:24.284291 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:24.285182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:24.285182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:24.285182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:24.285182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:24.285182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:24.285182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:24.285182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:24.285182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:24.285182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:24.285182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:24.285182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:24.285182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:24.285182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:24.285182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:24.285182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:24.285182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:24.285182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:24.286029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:24.286029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:24.286029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:24.286029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 280/565] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:24.286285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d100f6a-adf1-4412-9318-7d9ba7501595 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:24.329154 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:24.344860 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac62660-06e2-4f1f-a177-d354a3fb28c9 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:24.379442 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6c6c5-0c32-458d-92a1-1d0fa136ccab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:24.380051 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6c6c5-0c32-458d-92a1-1d0fa136ccab None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:24.380336 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6c6c5-0c32-458d-92a1-1d0fa136ccab None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:24.380609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6c6c5-0c32-458d-92a1-1d0fa136ccab None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:24.381264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6c6c5-0c32-458d-92a1-1d0fa136ccab None admin] RBAC: Adding query filter params (name=service) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:24.381571 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6c6c5-0c32-458d-92a1-1d0fa136ccab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:24.382964 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6c6c5-0c32-458d-92a1-1d0fa136ccab None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:24.391160 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 286/566] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 1 16:21:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:24.391936 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6c6c5-0c32-458d-92a1-1d0fa136ccab None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:24.392374 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d100f6a-adf1-4412-9318-7d9ba7501595 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:24.425900 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafc131-5ca2-401e-8908-313124e78cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:24.426550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafc131-5ca2-401e-8908-313124e78cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:24.426834 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafc131-5ca2-401e-8908-313124e78cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:24.427106 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafc131-5ca2-401e-8908-313124e78cd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:24.437387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebafc131-5ca2-401e-8908-313124e78cd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:24.438841 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebafc131-5ca2-401e-8908-313124e78cd4 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:24.452354 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:24.453079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-ebafc131-5ca2-401e-8908-313124e78cd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:24.453079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:24.453079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:24.453079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:24.453079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:24.453079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:24.453079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:24.453079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:24.453079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:24.453079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:24.453079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:24.453079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:24.453079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:24.453079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:24.453079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:24.453079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:24.454508 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:24.454508 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:24.454508 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:24.454508 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:24.454508 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:24.454508 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:24.454508 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:24.454508 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:24.454508 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:24.454508 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:24.454508 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:24.454508 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:24.454508 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:24.454508 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:24.454508 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:24.454508 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:24.454508 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:24.455366 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:24.455366 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:24.455366 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:24.455366 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 281/567] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:24.455366 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafc131-5ca2-401e-8908-313124e78cd4 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:24.493221 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6c6c5-0c32-458d-92a1-1d0fa136ccab None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:24.526011 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0bb18-f24c-4968-98f5-0d6727aee53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:24.526632 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0bb18-f24c-4968-98f5-0d6727aee53c None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:24.527447 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0bb18-f24c-4968-98f5-0d6727aee53c None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:24.527720 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0bb18-f24c-4968-98f5-0d6727aee53c None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:24.528353 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c0bb18-f24c-4968-98f5-0d6727aee53c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:24.528652 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c0bb18-f24c-4968-98f5-0d6727aee53c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:24.530010 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c0bb18-f24c-4968-98f5-0d6727aee53c None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:24.541710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 287/568] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:24.542463 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0bb18-f24c-4968-98f5-0d6727aee53c None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:24.542930 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:24.555270 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafc131-5ca2-401e-8908-313124e78cd4 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:24.588076 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada3969a-6ef8-4a05-8362-8c30544e6608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:24.588673 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3969a-6ef8-4a05-8362-8c30544e6608 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:24.588956 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3969a-6ef8-4a05-8362-8c30544e6608 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:24.589805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3969a-6ef8-4a05-8362-8c30544e6608 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:24.590485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada3969a-6ef8-4a05-8362-8c30544e6608 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:24.590787 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada3969a-6ef8-4a05-8362-8c30544e6608 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:24.592135 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada3969a-6ef8-4a05-8362-8c30544e6608 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:24.603449 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 282/569] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:24.604212 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada3969a-6ef8-4a05-8362-8c30544e6608 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:24.644646 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0bb18-f24c-4968-98f5-0d6727aee53c None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:24.666573 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:24.680362 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672c909-4259-4c52-ae2e-9483d57df90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:24.680986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672c909-4259-4c52-ae2e-9483d57df90b None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:24.681266 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672c909-4259-4c52-ae2e-9483d57df90b None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:24.681538 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672c909-4259-4c52-ae2e-9483d57df90b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:24.691430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8672c909-4259-4c52-ae2e-9483d57df90b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:24.696187 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-8672c909-4259-4c52-ae2e-9483d57df90b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:24.696187 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:24.696187 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:24.696187 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:24.696187 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:24.696187 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:24.696187 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:24.696187 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:24.696187 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:24.696187 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:24.696187 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:24.696187 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:24.696187 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:24.696187 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:24.696187 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:24.696187 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:24.697068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:24.697068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:24.697068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:24.697068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:24.697068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:24.697068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:24.697068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:24.697068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:24.697068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:24.697068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:24.697068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:24.697068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:24.697068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:24.697068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:24.697068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:24.697068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:24.697863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:24.697863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:24.697863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:24.697863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:24.697863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:24.697863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:24.697863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:24.697863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:24.697863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:24.697863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:24.697863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:24.697863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:24.697863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 288/570] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 1 16:21:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:24.699096 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672c909-4259-4c52-ae2e-9483d57df90b None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:24.702707 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada3969a-6ef8-4a05-8362-8c30544e6608 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:24.736174 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aefece-60c8-450a-b217-1fcb35941d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:24.736797 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aefece-60c8-450a-b217-1fcb35941d94 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:24.737075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aefece-60c8-450a-b217-1fcb35941d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:24.737345 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aefece-60c8-450a-b217-1fcb35941d94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:24.748228 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aefece-60c8-450a-b217-1fcb35941d94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:24.749598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aefece-60c8-450a-b217-1fcb35941d94 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:24.764538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-45aefece-60c8-450a-b217-1fcb35941d94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:24.764538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:24.764538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:24.764538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:24.764538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:24.764538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:24.764538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:24.764538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:24.764538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:24.764538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:24.764538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:24.764538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:24.764538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:24.764538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:24.764538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:24.764538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:24.765565 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:24.765565 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:24.765565 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:24.765565 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:24.765565 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:24.765565 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:24.765565 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:24.765565 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:24.765565 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:24.765565 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:24.765565 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:24.765565 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:24.765565 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:24.765565 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:24.765565 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:24.765565 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:24.765565 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:24.766432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:24.766432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:24.766432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:24.766432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 283/571] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:24.766432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aefece-60c8-450a-b217-1fcb35941d94 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:24.768848 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:24.801157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672c909-4259-4c52-ae2e-9483d57df90b None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:24.834526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149131f4-fee3-4f25-a481-b2a02a0d44cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:24.835125 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149131f4-fee3-4f25-a481-b2a02a0d44cc None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:24.835405 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149131f4-fee3-4f25-a481-b2a02a0d44cc None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:24.835678 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149131f4-fee3-4f25-a481-b2a02a0d44cc None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:24.836307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149131f4-fee3-4f25-a481-b2a02a0d44cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:24.836607 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149131f4-fee3-4f25-a481-b2a02a0d44cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:24.837970 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149131f4-fee3-4f25-a481-b2a02a0d44cc None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:24.848758 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 289/572] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:24.849531 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149131f4-fee3-4f25-a481-b2a02a0d44cc None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:24.871513 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aefece-60c8-450a-b217-1fcb35941d94 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:24.892246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:24.905264 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161435f-aa66-4111-b905-82862bdf97c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:24.905888 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161435f-aa66-4111-b905-82862bdf97c6 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:24.906169 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161435f-aa66-4111-b905-82862bdf97c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:24.906470 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161435f-aa66-4111-b905-82862bdf97c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:24.912883 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d161435f-aa66-4111-b905-82862bdf97c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:24.914316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d161435f-aa66-4111-b905-82862bdf97c6 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:24.925142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-d161435f-aa66-4111-b905-82862bdf97c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:24.925142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:24.925142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:24.925142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:24.925142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:24.925142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:24.925142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:24.925142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:24.925142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:24.925142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:24.925142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:24.925142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:24.925142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:21:24.925142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:21:24.925142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:21:24.926098 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:21:24.926098 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:24.926098 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:24.926098 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:24.926098 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:24.926098 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:24.926098 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:24.926098 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:24.926098 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:24.926098 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:24.926098 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:24.926098 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:24.926098 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:24.926098 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:24.926098 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:24.926098 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:24.926098 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:24.927395 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:21:24.927395 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:21:24.927395 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:21:24.927395 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:21:24.927395 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:21:24.927395 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:21:24.927395 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:24.927395 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:24.927395 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 284/573] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:24.927395 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161435f-aa66-4111-b905-82862bdf97c6 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:24.950334 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149131f4-fee3-4f25-a481-b2a02a0d44cc None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:24.982493 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b3413-150d-4917-9736-14add07f6263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:24.983117 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b3413-150d-4917-9736-14add07f6263 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:24.983400 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b3413-150d-4917-9736-14add07f6263 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:24.983695 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b3413-150d-4917-9736-14add07f6263 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:24.986803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:24.991243 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54b3413-150d-4917-9736-14add07f6263 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:24.992683 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54b3413-150d-4917-9736-14add07f6263 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:25.004302 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-e54b3413-150d-4917-9736-14add07f6263 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 16:21:25.004302 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:25.004302 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:25.004302 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:25.004302 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:25.004302 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:25.004302 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:25.004302 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:25.004302 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:25.004302 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:25.004302 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:25.004302 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:25.004302 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:21:25.004302 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:21:25.004302 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:21:25.004302 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:21:25.005383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:25.005383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:25.005383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:21:25.005383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:25.005383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:21:25.005383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:25.005383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:25.005383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:25.005383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:25.005383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:25.005383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:25.005383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:25.005383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:25.005383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:25.005383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:25.005383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:25.005383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:25.005383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:25.006510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:25.006510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:25.006510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:21:25.006510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:21:25.006510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:21:25.006510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:21:25.006510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:21:25.006510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:21:25.006510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 16:21:25.006510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:25.006510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 290/574] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:21:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:25.006510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b3413-150d-4917-9736-14add07f6263 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:25.028602 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161435f-aa66-4111-b905-82862bdf97c6 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:25.061704 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f2645-f497-4049-91f1-8c3b38ed0c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:25.062341 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f2645-f497-4049-91f1-8c3b38ed0c2d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:25.062625 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f2645-f497-4049-91f1-8c3b38ed0c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:25.062901 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f2645-f497-4049-91f1-8c3b38ed0c2d None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:25.063554 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091f2645-f497-4049-91f1-8c3b38ed0c2d None admin] RBAC: Adding query filter params (name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:25.063861 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091f2645-f497-4049-91f1-8c3b38ed0c2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:25.065206 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091f2645-f497-4049-91f1-8c3b38ed0c2d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:25.072908 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 285/575] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 1 16:21:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:25.074310 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f2645-f497-4049-91f1-8c3b38ed0c2d None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:25.105398 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b3413-150d-4917-9736-14add07f6263 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:25.106993 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:25.141284 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e24e26-cb00-4147-b453-23bf60fa8c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:25.141890 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e24e26-cb00-4147-b453-23bf60fa8c83 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:25.142170 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e24e26-cb00-4147-b453-23bf60fa8c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:25.142470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e24e26-cb00-4147-b453-23bf60fa8c83 None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:25.143098 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e24e26-cb00-4147-b453-23bf60fa8c83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:25.143400 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e24e26-cb00-4147-b453-23bf60fa8c83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:25.144758 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e24e26-cb00-4147-b453-23bf60fa8c83 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:25.156553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 291/576] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:25.157440 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e24e26-cb00-4147-b453-23bf60fa8c83 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:25.180618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f2645-f497-4049-91f1-8c3b38ed0c2d None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:25.203435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:25.214568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072bf26-ed33-4234-9faa-69beec602da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:25.215203 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072bf26-ed33-4234-9faa-69beec602da4 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:25.215487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072bf26-ed33-4234-9faa-69beec602da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:25.215763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072bf26-ed33-4234-9faa-69beec602da4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:25.223327 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b072bf26-ed33-4234-9faa-69beec602da4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:25.224744 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b072bf26-ed33-4234-9faa-69beec602da4 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:25.236064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-b072bf26-ed33-4234-9faa-69beec602da4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:21:25.236064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:25.236064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:25.236064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:25.236064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:25.236064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:25.236064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:25.236064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:25.236064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:25.236064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:25.236064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:25.236064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:25.236064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:21:25.236064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:21:25.236064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:21:25.236064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:21:25.237142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:25.237142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:25.237142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:21:25.237142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:25.237142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:21:25.237142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:25.237142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:25.237142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:25.237142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:25.237142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:25.237142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:25.237142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:25.237142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:25.237142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:25.237142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:25.237142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:25.237142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:25.237142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:25.242982 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:25.242982 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:25.242982 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:21:25.242982 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:21:25.242982 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:21:25.242982 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:21:25.242982 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:21:25.242982 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:21:25.242982 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:21:25.242982 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:25.242982 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 286/577] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:21:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:25.242982 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072bf26-ed33-4234-9faa-69beec602da4 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:25.256378 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e24e26-cb00-4147-b453-23bf60fa8c83 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:25.290795 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e5869-dbb4-474b-99f2-3dd294f159d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:25.291398 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e5869-dbb4-474b-99f2-3dd294f159d6 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:25.291682 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e5869-dbb4-474b-99f2-3dd294f159d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:25.291957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e5869-dbb4-474b-99f2-3dd294f159d6 None admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:25.293186 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e5869-dbb4-474b-99f2-3dd294f159d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:25.293489 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e5869-dbb4-474b-99f2-3dd294f159d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:25.294945 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e5869-dbb4-474b-99f2-3dd294f159d6 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:25.312625 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 292/578] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:21:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:25.313411 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e5869-dbb4-474b-99f2-3dd294f159d6 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:25.337150 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:25.348452 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072bf26-ed33-4234-9faa-69beec602da4 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:25.382857 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85733837-4e48-4c98-9fb0-b690f548935c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:25.383459 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85733837-4e48-4c98-9fb0-b690f548935c None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:25.383737 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85733837-4e48-4c98-9fb0-b690f548935c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:25.384011 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85733837-4e48-4c98-9fb0-b690f548935c None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:25.384728 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85733837-4e48-4c98-9fb0-b690f548935c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:25.385030 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85733837-4e48-4c98-9fb0-b690f548935c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:25.386461 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85733837-4e48-4c98-9fb0-b690f548935c None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:25.403178 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 287/579] 10.222.0.18 () {56 vars in 1137 bytes} [Thu Sep 1 16:21:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:25.403980 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85733837-4e48-4c98-9fb0-b690f548935c None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:25.419259 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e5869-dbb4-474b-99f2-3dd294f159d6 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:25.446566 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:25.453391 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46ecc5-b8e2-4a91-8756-d5ebf40fcb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:25.454107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46ecc5-b8e2-4a91-8756-d5ebf40fcb12 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:25.454430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46ecc5-b8e2-4a91-8756-d5ebf40fcb12 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:25.454709 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46ecc5-b8e2-4a91-8756-d5ebf40fcb12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:25.464588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b46ecc5-b8e2-4a91-8756-d5ebf40fcb12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:25.466044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b46ecc5-b8e2-4a91-8756-d5ebf40fcb12 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:25.480828 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-3b46ecc5-b8e2-4a91-8756-d5ebf40fcb12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:25.480828 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:25.480828 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:25.480828 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:25.480828 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:25.480828 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:25.480828 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:25.480828 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:25.480828 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:25.480828 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:25.480828 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:25.480828 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:25.480828 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:25.480828 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:25.480828 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:25.480828 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:25.481773 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:25.481773 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:25.481773 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:25.481773 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:25.481773 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:25.481773 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:25.481773 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:25.481773 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:25.481773 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:25.481773 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:25.481773 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:25.481773 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:25.481773 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:25.481773 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:25.481773 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:25.481773 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:25.481773 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:25.482908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:25.482908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:25.482908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:25.482908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 293/580] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:25.482908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46ecc5-b8e2-4a91-8756-d5ebf40fcb12 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:25.510484 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85733837-4e48-4c98-9fb0-b690f548935c None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:25.544661 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e1907-76e6-436a-ae99-cc1d7cf2fc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:25.545288 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e1907-76e6-436a-ae99-cc1d7cf2fc6f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:25.545571 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e1907-76e6-436a-ae99-cc1d7cf2fc6f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:25.545844 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e1907-76e6-436a-ae99-cc1d7cf2fc6f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:25.553082 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e1907-76e6-436a-ae99-cc1d7cf2fc6f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:25.554571 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e1907-76e6-436a-ae99-cc1d7cf2fc6f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:25.555698 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:25.565941 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-ef2e1907-76e6-436a-ae99-cc1d7cf2fc6f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:21:25.565941 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:25.565941 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:25.565941 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:25.565941 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:25.565941 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:25.565941 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:25.565941 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:25.565941 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:25.565941 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:25.565941 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:25.565941 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:25.565941 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:21:25.565941 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:21:25.565941 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:21:25.565941 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:21:25.566949 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:25.566949 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:25.566949 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:21:25.566949 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:25.566949 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:21:25.566949 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:25.566949 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:25.566949 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:25.566949 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:25.566949 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:25.566949 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:25.566949 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:25.566949 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:25.566949 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:25.566949 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:25.566949 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:25.566949 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:25.566949 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:25.568251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:25.568251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:25.568251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:21:25.568251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:21:25.568251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:21:25.568251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:21:25.568251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:21:25.568251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:21:25.568251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:21:25.568251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:25.568251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 288/581] 10.222.0.18 () {56 vars in 1093 bytes} [Thu Sep 1 16:21:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:25.568251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e1907-76e6-436a-ae99-cc1d7cf2fc6f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:25.582477 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46ecc5-b8e2-4a91-8756-d5ebf40fcb12 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:25.616505 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47141d4-5577-454f-a3b7-7610d0eccf6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:25.617187 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47141d4-5577-454f-a3b7-7610d0eccf6c None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:25.617475 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47141d4-5577-454f-a3b7-7610d0eccf6c None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:25.617752 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47141d4-5577-454f-a3b7-7610d0eccf6c None admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:25.618500 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47141d4-5577-454f-a3b7-7610d0eccf6c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:25.618810 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47141d4-5577-454f-a3b7-7610d0eccf6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:25.620292 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47141d4-5577-454f-a3b7-7610d0eccf6c None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:25.638113 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 294/582] 10.222.0.18 () {56 vars in 1137 bytes} [Thu Sep 1 16:21:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:25.638930 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47141d4-5577-454f-a3b7-7610d0eccf6c None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:25.668119 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e1907-76e6-436a-ae99-cc1d7cf2fc6f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:25.669655 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:25.702463 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95201d-aa54-4851-bdc4-519e686c0985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:25.703067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95201d-aa54-4851-bdc4-519e686c0985 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:25.703342 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95201d-aa54-4851-bdc4-519e686c0985 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:25.703638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95201d-aa54-4851-bdc4-519e686c0985 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:25.704354 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b95201d-aa54-4851-bdc4-519e686c0985 None admin] RBAC: Adding query filter params (role.id=b6727ed962fa41109b0a2d6851551f49, scope.project.id=36ce80c4e0d64562a9ab3961be76fad9, user.id=8137f6b2ced040bd9752239c7e987f03) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:25.704657 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b95201d-aa54-4851-bdc4-519e686c0985 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:25.706064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b95201d-aa54-4851-bdc4-519e686c0985 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:25.711275 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 289/583] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:21:25 2022] GET /identity/v3/role_assignments?user.id=8137f6b2ced040bd9752239c7e987f03&scope.project.id=36ce80c4e0d64562a9ab3961be76fad9&role.id=b6727ed962fa41109b0a2d6851551f49 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:25.712038 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95201d-aa54-4851-bdc4-519e686c0985 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:25.749435 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47141d4-5577-454f-a3b7-7610d0eccf6c None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:25.784542 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:25.791441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4b715a-85b3-41ae-8eb5-3a6eb6e31259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:25.792087 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b715a-85b3-41ae-8eb5-3a6eb6e31259 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:25.792952 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b715a-85b3-41ae-8eb5-3a6eb6e31259 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:25.793227 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b715a-85b3-41ae-8eb5-3a6eb6e31259 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:25.804997 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4b715a-85b3-41ae-8eb5-3a6eb6e31259 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:25.806652 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4b715a-85b3-41ae-8eb5-3a6eb6e31259 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:25.822897 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-3d4b715a-85b3-41ae-8eb5-3a6eb6e31259 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:25.822897 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:25.822897 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:25.822897 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:25.822897 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:25.822897 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:25.822897 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:25.822897 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:25.822897 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:25.822897 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:25.822897 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:25.822897 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:25.822897 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:25.822897 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:25.822897 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:25.822897 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:25.823981 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:25.823981 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:25.823981 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:25.823981 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:25.823981 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:25.823981 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:25.823981 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:25.823981 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:25.823981 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:25.823981 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:25.823981 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:25.823981 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:25.823981 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:25.823981 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:25.823981 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:25.823981 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:25.823981 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:25.830035 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:25.830035 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:25.830035 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:25.830206 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95201d-aa54-4851-bdc4-519e686c0985 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:25.830338 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 295/584] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:25.830338 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4b715a-85b3-41ae-8eb5-3a6eb6e31259 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:25.872688 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd23c34-7b3b-426e-9ebc-335c7a313a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:25.873739 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd23c34-7b3b-426e-9ebc-335c7a313a92 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:25.874035 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd23c34-7b3b-426e-9ebc-335c7a313a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:25.874443 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd23c34-7b3b-426e-9ebc-335c7a313a92 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:25.875100 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd23c34-7b3b-426e-9ebc-335c7a313a92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:25.875781 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd23c34-7b3b-426e-9ebc-335c7a313a92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:25.877485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd23c34-7b3b-426e-9ebc-335c7a313a92 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:25.894394 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 290/585] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:25.894394 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd23c34-7b3b-426e-9ebc-335c7a313a92 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:25.920597 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:25.943375 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4b715a-85b3-41ae-8eb5-3a6eb6e31259 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:25.979118 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ba16b-ed24-4be0-9ba2-d3ad300a0c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:25.981936 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ba16b-ed24-4be0-9ba2-d3ad300a0c91 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:25.981936 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ba16b-ed24-4be0-9ba2-d3ad300a0c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:25.981936 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ba16b-ed24-4be0-9ba2-d3ad300a0c91 None admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:25.981936 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ba16b-ed24-4be0-9ba2-d3ad300a0c91 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:25.983861 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ba16b-ed24-4be0-9ba2-d3ad300a0c91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:25.985333 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ba16b-ed24-4be0-9ba2-d3ad300a0c91 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:26.004037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 296/586] 10.222.0.18 () {56 vars in 1133 bytes} [Thu Sep 1 16:21:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:26.004659 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd23c34-7b3b-426e-9ebc-335c7a313a92 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:26.004855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ba16b-ed24-4be0-9ba2-d3ad300a0c91 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:26.043850 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950ae1f-98a7-464c-80b8-3fbf7b21dd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:26.044413 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950ae1f-98a7-464c-80b8-3fbf7b21dd0f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:26.044696 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950ae1f-98a7-464c-80b8-3fbf7b21dd0f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:26.044973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950ae1f-98a7-464c-80b8-3fbf7b21dd0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:26.054186 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:26.056439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7950ae1f-98a7-464c-80b8-3fbf7b21dd0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:26.058144 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7950ae1f-98a7-464c-80b8-3fbf7b21dd0f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:26.073107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-7950ae1f-98a7-464c-80b8-3fbf7b21dd0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:26.073107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:26.073107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:26.073107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:26.073107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:26.073107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:26.073107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:26.073107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:26.073107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:26.073107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:26.073107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:26.073107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:26.073107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:26.073107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:26.073107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:26.073107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:26.074645 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:26.074645 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:26.074645 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:26.074645 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:26.074645 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:26.074645 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:26.074645 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:26.074645 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:26.074645 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:26.074645 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:26.074645 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:26.074645 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:26.074645 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:26.074645 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:26.074645 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:26.074645 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:26.074645 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:26.075544 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:26.075544 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:26.075544 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:26.075544 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 291/587] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:26.075544 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950ae1f-98a7-464c-80b8-3fbf7b21dd0f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:26.120954 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ba16b-ed24-4be0-9ba2-d3ad300a0c91 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:26.158807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9c95e-6124-4a2a-844e-94ab640839e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:26.159772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9c95e-6124-4a2a-844e-94ab640839e2 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:26.160055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9c95e-6124-4a2a-844e-94ab640839e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:26.160325 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9c95e-6124-4a2a-844e-94ab640839e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:26.160792 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:26.161257 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f9c95e-6124-4a2a-844e-94ab640839e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:26.161561 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f9c95e-6124-4a2a-844e-94ab640839e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:26.163247 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f9c95e-6124-4a2a-844e-94ab640839e2 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:26.179164 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 297/588] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:26.179164 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9c95e-6124-4a2a-844e-94ab640839e2 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:26.187676 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950ae1f-98a7-464c-80b8-3fbf7b21dd0f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:26.227753 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da85ce55-f02a-4484-8175-65931415e81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:26.228594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85ce55-f02a-4484-8175-65931415e81a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:26.228880 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85ce55-f02a-4484-8175-65931415e81a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:26.229321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85ce55-f02a-4484-8175-65931415e81a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:26.240715 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da85ce55-f02a-4484-8175-65931415e81a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:26.242630 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da85ce55-f02a-4484-8175-65931415e81a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:26.258082 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-da85ce55-f02a-4484-8175-65931415e81a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 16:21:26.258082 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:26.258082 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:26.258082 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:26.258082 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:26.258082 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:26.258082 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:26.258082 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:26.258082 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:26.258082 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:26.258082 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:26.258082 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:26.258082 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:21:26.258082 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:21:26.258082 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:21:26.258082 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:21:26.259238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:26.259238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:26.259238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:21:26.259238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:26.259238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:21:26.259238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:26.259238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:26.259238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:26.259238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:26.259238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:26.259238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:26.259238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:26.259238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:26.259238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:26.259238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:26.259238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:26.259238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:26.259238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:26.260466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:26.260466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:26.260466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:21:26.260466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:21:26.260466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:21:26.260466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:21:26.260466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:21:26.260466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:21:26.260466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 16:21:26.260466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:26.262564 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 292/589] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 1 16:21:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:26.263535 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da85ce55-f02a-4484-8175-65931415e81a None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:26.291186 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9c95e-6124-4a2a-844e-94ab640839e2 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:26.293841 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:26.328983 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a424847-d3da-4bd3-a16e-94a16a92e2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:26.329836 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a424847-d3da-4bd3-a16e-94a16a92e2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:26.330117 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a424847-d3da-4bd3-a16e-94a16a92e2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:26.330426 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a424847-d3da-4bd3-a16e-94a16a92e2c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:26.341082 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a424847-d3da-4bd3-a16e-94a16a92e2c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:26.342569 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a424847-d3da-4bd3-a16e-94a16a92e2c1 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:26.357750 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-0a424847-d3da-4bd3-a16e-94a16a92e2c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:26.357750 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:26.357750 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:26.357750 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:26.357750 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:26.357750 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:26.357750 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:26.357750 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:26.357750 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:26.357750 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:26.357750 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:26.357750 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:26.357750 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:26.357750 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:26.357750 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:26.357750 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:26.358890 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:26.358890 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:26.358890 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:26.358890 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:26.358890 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:26.358890 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:26.358890 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:26.358890 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:26.358890 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:26.358890 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:26.358890 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:26.358890 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:26.358890 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:26.358890 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:26.358890 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:26.358890 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:26.358890 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:26.360427 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:26.360427 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:26.360427 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:26.366798 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 298/590] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:26.366798 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a424847-d3da-4bd3-a16e-94a16a92e2c1 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:26.377151 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da85ce55-f02a-4484-8175-65931415e81a None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:26.412874 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27351a3-ca60-4c14-914b-2e1f33c0cd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:26.413482 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27351a3-ca60-4c14-914b-2e1f33c0cd7d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:26.413774 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27351a3-ca60-4c14-914b-2e1f33c0cd7d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:26.414051 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27351a3-ca60-4c14-914b-2e1f33c0cd7d None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:26.414725 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27351a3-ca60-4c14-914b-2e1f33c0cd7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:26.415035 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27351a3-ca60-4c14-914b-2e1f33c0cd7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:26.416424 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27351a3-ca60-4c14-914b-2e1f33c0cd7d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:26.423285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:26.432781 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 293/591] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:26.434200 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27351a3-ca60-4c14-914b-2e1f33c0cd7d None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:26.473428 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a424847-d3da-4bd3-a16e-94a16a92e2c1 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:26.509329 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcfd763-b480-4a6c-b4c1-da8301b4752b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:26.509965 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcfd763-b480-4a6c-b4c1-da8301b4752b None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:26.510272 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcfd763-b480-4a6c-b4c1-da8301b4752b None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:26.510558 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcfd763-b480-4a6c-b4c1-da8301b4752b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:26.517918 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcfd763-b480-4a6c-b4c1-da8301b4752b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:26.519420 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcfd763-b480-4a6c-b4c1-da8301b4752b None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:26.530895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-1bcfd763-b480-4a6c-b4c1-da8301b4752b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:26.530895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:26.530895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:26.530895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:26.530895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:26.530895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:26.530895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:26.530895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:26.530895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:26.530895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:26.530895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:26.530895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:26.530895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:21:26.530895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:21:26.530895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:21:26.531775 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:21:26.531775 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:26.531775 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:26.531775 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:26.531775 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:26.531775 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:26.531775 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:26.531775 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:26.531775 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:26.531775 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:26.531775 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:26.531775 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:26.531775 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:26.531775 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:26.531775 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:26.531775 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:26.531775 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:26.532635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:21:26.532635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:21:26.532635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:21:26.532635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:21:26.532635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:21:26.532635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:21:26.532635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:26.532635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:26.532635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 299/592] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:26.533105 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcfd763-b480-4a6c-b4c1-da8301b4752b None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:26.549738 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27351a3-ca60-4c14-914b-2e1f33c0cd7d None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:26.551928 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:26.588422 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351b9df-e738-4173-adcf-b294346ff733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:26.588964 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351b9df-e738-4173-adcf-b294346ff733 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:26.589326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351b9df-e738-4173-adcf-b294346ff733 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:26.589609 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351b9df-e738-4173-adcf-b294346ff733 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:26.590387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351b9df-e738-4173-adcf-b294346ff733 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:26.590702 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351b9df-e738-4173-adcf-b294346ff733 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:26.592125 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351b9df-e738-4173-adcf-b294346ff733 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:26.609281 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 294/593] 10.222.0.18 () {56 vars in 1139 bytes} [Thu Sep 1 16:21:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:26.610076 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351b9df-e738-4173-adcf-b294346ff733 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:26.649989 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcfd763-b480-4a6c-b4c1-da8301b4752b None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:26.671812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:26.687619 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e93b2-46d0-4175-9c59-8666f09cf5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:26.687619 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e93b2-46d0-4175-9c59-8666f09cf5e9 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:26.688638 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e93b2-46d0-4175-9c59-8666f09cf5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:26.688920 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e93b2-46d0-4175-9c59-8666f09cf5e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:26.689786 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008e93b2-46d0-4175-9c59-8666f09cf5e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:26.689907 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008e93b2-46d0-4175-9c59-8666f09cf5e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:26.691261 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008e93b2-46d0-4175-9c59-8666f09cf5e9 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:26.707578 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 300/594] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:26.707578 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e93b2-46d0-4175-9c59-8666f09cf5e9 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:26.725899 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351b9df-e738-4173-adcf-b294346ff733 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:26.763631 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252f45d-3590-4782-9ad3-d73dd66e43bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:26.764287 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252f45d-3590-4782-9ad3-d73dd66e43bb None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:26.764572 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252f45d-3590-4782-9ad3-d73dd66e43bb None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:26.764871 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252f45d-3590-4782-9ad3-d73dd66e43bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:26.772685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0252f45d-3590-4782-9ad3-d73dd66e43bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:26.774284 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0252f45d-3590-4782-9ad3-d73dd66e43bb None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:26.785622 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-0252f45d-3590-4782-9ad3-d73dd66e43bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:26.785622 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:26.785622 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:26.785622 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:26.785622 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:26.785622 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:26.785622 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:26.785622 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:26.785622 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:26.785622 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:26.785622 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:26.785622 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:26.785622 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:21:26.785622 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:21:26.785622 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:21:26.786694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:21:26.786694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:26.786694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:26.786694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:26.786694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:26.786694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:26.786694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:26.786694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:26.786694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:26.786694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:26.786694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:26.786694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:26.786694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:26.786694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:26.786694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:26.786694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:26.786694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:26.787770 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:21:26.787770 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:21:26.787770 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:21:26.787770 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:21:26.787770 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:21:26.787770 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:21:26.787770 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:26.787770 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:26.788960 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 295/595] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:26.789196 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:26.789733 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252f45d-3590-4782-9ad3-d73dd66e43bb None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:26.820448 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e93b2-46d0-4175-9c59-8666f09cf5e9 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:26.857821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86684dd6-9120-4e7f-84a3-8946a551974f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:26.858731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86684dd6-9120-4e7f-84a3-8946a551974f None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:26.859162 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86684dd6-9120-4e7f-84a3-8946a551974f None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:26.859585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86684dd6-9120-4e7f-84a3-8946a551974f None admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:26.860524 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86684dd6-9120-4e7f-84a3-8946a551974f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:26.861007 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86684dd6-9120-4e7f-84a3-8946a551974f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:26.862837 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86684dd6-9120-4e7f-84a3-8946a551974f None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:26.876596 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 301/596] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:21:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:26.877729 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86684dd6-9120-4e7f-84a3-8946a551974f None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:26.900863 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252f45d-3590-4782-9ad3-d73dd66e43bb None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:26.917107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:26.936829 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a4c96-2cae-4909-888e-b1ed4f15389f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:26.937555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a4c96-2cae-4909-888e-b1ed4f15389f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:26.937911 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a4c96-2cae-4909-888e-b1ed4f15389f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:26.938284 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a4c96-2cae-4909-888e-b1ed4f15389f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:26.950027 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a4c96-2cae-4909-888e-b1ed4f15389f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:26.951552 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a4c96-2cae-4909-888e-b1ed4f15389f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:26.965321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-b21a4c96-2cae-4909-888e-b1ed4f15389f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:26.965321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:26.965321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:26.965321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:26.965321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:26.965321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:26.965321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:26.965321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:26.965321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:26.965321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:26.965321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:26.965321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:26.965321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:26.965321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:26.965321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:26.965321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:26.966405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:26.966405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:26.966405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:26.966405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:26.966405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:26.966405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:26.966405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:26.966405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:26.966405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:26.966405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:26.966405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:26.966405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:26.966405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:26.966405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:26.966405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:26.966405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:26.966405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:26.967312 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:26.967312 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:26.967312 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:26.967312 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 296/597] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:26.967605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a4c96-2cae-4909-888e-b1ed4f15389f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:26.988738 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86684dd6-9120-4e7f-84a3-8946a551974f None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:27.027040 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69488f9-9c3b-4267-bde8-812eacc15512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:27.027785 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69488f9-9c3b-4267-bde8-812eacc15512 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:27.028143 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69488f9-9c3b-4267-bde8-812eacc15512 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:27.028494 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69488f9-9c3b-4267-bde8-812eacc15512 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:27.031246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:27.036636 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69488f9-9c3b-4267-bde8-812eacc15512 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:27.038385 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69488f9-9c3b-4267-bde8-812eacc15512 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:27.050325 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-f69488f9-9c3b-4267-bde8-812eacc15512 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:27.050325 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:27.050325 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:27.050325 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:27.050325 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:27.050325 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:27.050325 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:27.050325 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:27.050325 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:27.050325 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:27.050325 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:27.050325 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:27.050325 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:21:27.050325 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:21:27.050325 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:21:27.051368 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:21:27.051368 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:27.051368 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:27.051368 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:27.051368 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:27.051368 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:27.051368 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:27.051368 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:27.051368 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:27.051368 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:27.051368 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:27.051368 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:27.051368 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:27.051368 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:27.051368 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:27.051368 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:27.051368 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:27.052461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:21:27.052461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:21:27.052461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:21:27.052461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:21:27.052461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:21:27.052461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:21:27.052461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:27.052461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:27.056300 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 302/598] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:27.057119 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69488f9-9c3b-4267-bde8-812eacc15512 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:27.084108 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a4c96-2cae-4909-888e-b1ed4f15389f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:27.121606 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3300df-4028-4bb7-911b-292196e49269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:27.122157 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3300df-4028-4bb7-911b-292196e49269 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:27.122461 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3300df-4028-4bb7-911b-292196e49269 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:27.122725 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3300df-4028-4bb7-911b-292196e49269 None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:27.123416 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3300df-4028-4bb7-911b-292196e49269 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:27.123726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3300df-4028-4bb7-911b-292196e49269 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:27.125148 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3300df-4028-4bb7-911b-292196e49269 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:27.137592 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 297/599] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:21:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:27.138200 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3300df-4028-4bb7-911b-292196e49269 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:27.173308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:27.175021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69488f9-9c3b-4267-bde8-812eacc15512 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:27.214635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d982c-5689-4353-92b6-106ca0997bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:27.215382 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d982c-5689-4353-92b6-106ca0997bb2 None admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:27.215730 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d982c-5689-4353-92b6-106ca0997bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:27.216010 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d982c-5689-4353-92b6-106ca0997bb2 None admin] PATH_INFO: `/v3/projects/36ce80c4e0d64562a9ab3961be76fad9/users/b09f6d550c9142bf956b225bf5172e42/roles/b6727ed962fa41109b0a2d6851551f49` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:27.232265 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5d982c-5689-4353-92b6-106ca0997bb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=36ce80c4e0d64562a9ab3961be76fad9, user_id=b09f6d550c9142bf956b225bf5172e42, role_id=b6727ed962fa41109b0a2d6851551f49)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:27.233721 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5d982c-5689-4353-92b6-106ca0997bb2 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:27.248352 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 303/600] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 1 16:21:27 2022] PUT /identity/v3/projects/36ce80c4e0d64562a9ab3961be76fad9/users/b09f6d550c9142bf956b225bf5172e42/roles/b6727ed962fa41109b0a2d6851551f49 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:21:27.249302 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d982c-5689-4353-92b6-106ca0997bb2 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:27.261359 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3300df-4028-4bb7-911b-292196e49269 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:27.305033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a003a05-faa1-4e2b-b771-bd08ba4a9925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:27.306029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a003a05-faa1-4e2b-b771-bd08ba4a9925 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:27.306502 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a003a05-faa1-4e2b-b771-bd08ba4a9925 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:27.306999 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a003a05-faa1-4e2b-b771-bd08ba4a9925 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:27.307865 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a003a05-faa1-4e2b-b771-bd08ba4a9925 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:27.308344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a003a05-faa1-4e2b-b771-bd08ba4a9925 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:27.308808 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:27.310932 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a003a05-faa1-4e2b-b771-bd08ba4a9925 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:27.326340 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 298/601] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:27.327708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a003a05-faa1-4e2b-b771-bd08ba4a9925 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:27.384924 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d982c-5689-4353-92b6-106ca0997bb2 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:27.433741 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7085d4b5-a8d8-4188-8de1-661f60af93a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:27.435101 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085d4b5-a8d8-4188-8de1-661f60af93a5 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:27.435606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085d4b5-a8d8-4188-8de1-661f60af93a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:27.436312 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085d4b5-a8d8-4188-8de1-661f60af93a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:27.437384 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7085d4b5-a8d8-4188-8de1-661f60af93a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:27.437807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7085d4b5-a8d8-4188-8de1-661f60af93a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:27.439669 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7085d4b5-a8d8-4188-8de1-661f60af93a5 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:27.441698 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:27.455749 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a003a05-faa1-4e2b-b771-bd08ba4a9925 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:27.460331 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 304/602] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:21:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:27.465706 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7085d4b5-a8d8-4188-8de1-661f60af93a5 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:27.498431 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74791ca4-f962-4a3d-ad33-75861fc68bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:27.499249 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74791ca4-f962-4a3d-ad33-75861fc68bd0 None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:27.499563 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74791ca4-f962-4a3d-ad33-75861fc68bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:27.499817 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74791ca4-f962-4a3d-ad33-75861fc68bd0 None admin] PATH_INFO: `/v3/projects/36ce80c4e0d64562a9ab3961be76fad9/users/b78747bb9140457896fe62d4f537592c/roles/b6727ed962fa41109b0a2d6851551f49` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:27.516166 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74791ca4-f962-4a3d-ad33-75861fc68bd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=36ce80c4e0d64562a9ab3961be76fad9, user_id=b78747bb9140457896fe62d4f537592c, role_id=b6727ed962fa41109b0a2d6851551f49)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:27.517688 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74791ca4-f962-4a3d-ad33-75861fc68bd0 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:27.532862 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 299/603] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 1 16:21:27 2022] PUT /identity/v3/projects/36ce80c4e0d64562a9ab3961be76fad9/users/b78747bb9140457896fe62d4f537592c/roles/b6727ed962fa41109b0a2d6851551f49 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:21:27.533802 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74791ca4-f962-4a3d-ad33-75861fc68bd0 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:27.608554 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7085d4b5-a8d8-4188-8de1-661f60af93a5 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:27.615585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:27.653022 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74791ca4-f962-4a3d-ad33-75861fc68bd0 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:27.659453 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa921821-789f-446c-aaed-ad9a60959aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:27.660117 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa921821-789f-446c-aaed-ad9a60959aa2 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:27.660406 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa921821-789f-446c-aaed-ad9a60959aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:27.660682 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa921821-789f-446c-aaed-ad9a60959aa2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:27.671604 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa921821-789f-446c-aaed-ad9a60959aa2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:27.673781 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa921821-789f-446c-aaed-ad9a60959aa2 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:27.690448 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-fa921821-789f-446c-aaed-ad9a60959aa2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:27.690448 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:27.690448 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:27.690448 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:27.690448 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:27.690448 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:27.690448 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:27.690448 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:27.690448 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:27.690448 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:27.690448 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:27.690448 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:27.690448 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:21:27.690448 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:21:27.690448 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:21:27.691373 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:21:27.691373 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:27.691373 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:27.691373 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:27.691373 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:27.691373 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:27.691373 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:27.691373 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:27.691373 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:27.691373 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:27.691373 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:27.691373 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:27.691373 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:27.691373 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:27.691373 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:27.691373 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:27.691373 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:27.692234 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:21:27.692234 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:21:27.692234 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:21:27.692234 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:21:27.692234 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:21:27.692234 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:21:27.692234 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:27.692234 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:27.692234 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 305/604] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:27.696381 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0279ae5-6634-494a-a1c5-fe986626fde5 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:27.696381 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0279ae5-6634-494a-a1c5-fe986626fde5 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:27.696381 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0279ae5-6634-494a-a1c5-fe986626fde5 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:27.696381 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 306/605] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:27.697358 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0279ae5-6634-494a-a1c5-fe986626fde5 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:27.702485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7fd72-84b3-46d7-be21-cebd4533f0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:27.703238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7fd72-84b3-46d7-be21-cebd4533f0e1 None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:27.703531 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7fd72-84b3-46d7-be21-cebd4533f0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:27.703810 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7fd72-84b3-46d7-be21-cebd4533f0e1 None admin] PATH_INFO: `/v3/projects/36ce80c4e0d64562a9ab3961be76fad9/users/278e6d6aed414c758fe3e9f4df6bf81e/roles/b6727ed962fa41109b0a2d6851551f49` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:27.723877 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd7fd72-84b3-46d7-be21-cebd4533f0e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=36ce80c4e0d64562a9ab3961be76fad9, user_id=278e6d6aed414c758fe3e9f4df6bf81e, role_id=b6727ed962fa41109b0a2d6851551f49)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:27.725344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd7fd72-84b3-46d7-be21-cebd4533f0e1 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:27.730135 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:27.747835 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 300/606] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 1 16:21:27 2022] PUT /identity/v3/projects/36ce80c4e0d64562a9ab3961be76fad9/users/278e6d6aed414c758fe3e9f4df6bf81e/roles/b6727ed962fa41109b0a2d6851551f49 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:21:27.749542 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df3477-fadb-45a0-a015-3e00cf908c03 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:27.749542 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df3477-fadb-45a0-a015-3e00cf908c03 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:27.750188 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df3477-fadb-45a0-a015-3e00cf908c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:27.775534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-21df3477-fadb-45a0-a015-3e00cf908c03 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:27.776976 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-21df3477-fadb-45a0-a015-3e00cf908c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:27.861132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0279ae5-6634-494a-a1c5-fe986626fde5 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:27.888503 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 301/607] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:21:27.913528 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cba258-c38c-47b6-bdac-6cba17b84570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:27.914121 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cba258-c38c-47b6-bdac-6cba17b84570 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:27.914499 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cba258-c38c-47b6-bdac-6cba17b84570 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:27.914778 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cba258-c38c-47b6-bdac-6cba17b84570 None admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:27.915483 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cba258-c38c-47b6-bdac-6cba17b84570 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:27.915848 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cba258-c38c-47b6-bdac-6cba17b84570 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:27.917399 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cba258-c38c-47b6-bdac-6cba17b84570 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:27.931063 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 307/608] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:21:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:27.937853 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df3477-fadb-45a0-a015-3e00cf908c03 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:28.047662 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df3477-fadb-45a0-a015-3e00cf908c03 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:28.066308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:28.086265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f95ad-d276-4c75-aff2-48767e94466f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:28.086982 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f95ad-d276-4c75-aff2-48767e94466f None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:28.087266 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f95ad-d276-4c75-aff2-48767e94466f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:28.087539 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f95ad-d276-4c75-aff2-48767e94466f None admin] PATH_INFO: `/v3/projects/36ce80c4e0d64562a9ab3961be76fad9/users/ffa0f2160a934906b59e6f845edb0776/roles/b6727ed962fa41109b0a2d6851551f49` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:28.103504 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f95ad-d276-4c75-aff2-48767e94466f None admin] RBAC: Authorizing `identity:create_grant(project_id=36ce80c4e0d64562a9ab3961be76fad9, user_id=ffa0f2160a934906b59e6f845edb0776, role_id=b6727ed962fa41109b0a2d6851551f49)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:28.105005 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f95ad-d276-4c75-aff2-48767e94466f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:28.128467 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 302/609] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 1 16:21:27 2022] PUT /identity/v3/projects/36ce80c4e0d64562a9ab3961be76fad9/users/ffa0f2160a934906b59e6f845edb0776/roles/b6727ed962fa41109b0a2d6851551f49 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:21:28.225618 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cba258-c38c-47b6-bdac-6cba17b84570 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:28.262684 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:28.348318 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cba258-c38c-47b6-bdac-6cba17b84570 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:28.385430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6857be-20a4-4046-bf27-545176130893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:28.386642 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6857be-20a4-4046-bf27-545176130893 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:28.386927 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6857be-20a4-4046-bf27-545176130893 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:28.387206 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6857be-20a4-4046-bf27-545176130893 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:28.398410 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6857be-20a4-4046-bf27-545176130893 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:28.405079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-ff6857be-20a4-4046-bf27-545176130893 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:28.405079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:28.405079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:28.405079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:28.405079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:28.405079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:28.405079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:28.405079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:28.405079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:28.405079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:28.405079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:28.405079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:28.405079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:28.405079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:28.405079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:28.405079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:28.405986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:28.405986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:28.405986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:28.405986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:28.405986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:28.405986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:28.405986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:28.405986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:28.405986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:28.405986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:28.405986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:28.405986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:28.405986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:28.405986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:28.405986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:28.405986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:28.406852 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:28.406852 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:28.406852 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:28.406852 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:28.406852 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:28.406852 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:28.406852 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:28.406852 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:28.406852 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:28.406852 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:28.406852 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:28.406852 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:28.406852 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 308/610] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 1 16:21:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:28.415316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f95ad-d276-4c75-aff2-48767e94466f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:28.573896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f95ad-d276-4c75-aff2-48767e94466f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:28.616057 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b11c205-2290-4fde-86a5-9d01cc972753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:28.616675 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11c205-2290-4fde-86a5-9d01cc972753 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:28.616962 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11c205-2290-4fde-86a5-9d01cc972753 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:28.617239 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11c205-2290-4fde-86a5-9d01cc972753 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:28.622788 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b11c205-2290-4fde-86a5-9d01cc972753 None admin] RBAC: Adding query filter params (name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:28.623115 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b11c205-2290-4fde-86a5-9d01cc972753 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:28.624495 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b11c205-2290-4fde-86a5-9d01cc972753 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:28.633107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 303/611] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 1 16:21:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:28.639309 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6857be-20a4-4046-bf27-545176130893 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:28.761633 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:28.778958 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6857be-20a4-4046-bf27-545176130893 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:28.821748 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9446676-a355-4382-a433-9ec60364a527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:28.822772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9446676-a355-4382-a433-9ec60364a527 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:28.823317 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9446676-a355-4382-a433-9ec60364a527 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:28.823873 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9446676-a355-4382-a433-9ec60364a527 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:28.835472 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9446676-a355-4382-a433-9ec60364a527 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:28.837170 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9446676-a355-4382-a433-9ec60364a527 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:28.857822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-f9446676-a355-4382-a433-9ec60364a527 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:28.857822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:28.857822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:28.857822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:28.857822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:28.857822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:28.857822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:28.857822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:28.857822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:28.857822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:28.857822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:28.857822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:28.857822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:28.857822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:28.857822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:28.857822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:28.858821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:28.858821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:28.858821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:28.858821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:28.858821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:28.858821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:28.858821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:28.858821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:28.858821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:28.858821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:28.858821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:28.858821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:28.858821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:28.858821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:28.858821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:28.858821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:28.858821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:28.859683 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:28.859683 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:28.859683 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:28.861762 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 309/612] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:28.867095 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b11c205-2290-4fde-86a5-9d01cc972753 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:28.936545 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:29.026491 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b11c205-2290-4fde-86a5-9d01cc972753 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:29.070048 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b88b5-86df-42ca-9d0a-cbf9844f9504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:29.071034 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b88b5-86df-42ca-9d0a-cbf9844f9504 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:29.071526 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b88b5-86df-42ca-9d0a-cbf9844f9504 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:29.072008 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b88b5-86df-42ca-9d0a-cbf9844f9504 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:29.072884 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b88b5-86df-42ca-9d0a-cbf9844f9504 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:29.073944 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b88b5-86df-42ca-9d0a-cbf9844f9504 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:29.075531 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b88b5-86df-42ca-9d0a-cbf9844f9504 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:29.093263 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 304/613] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:21:29.098969 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9446676-a355-4382-a433-9ec60364a527 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:29.207781 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9446676-a355-4382-a433-9ec60364a527 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:29.248631 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7db5a1-54e5-4931-b9cf-779345109add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:29.249601 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7db5a1-54e5-4931-b9cf-779345109add None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:29.250106 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7db5a1-54e5-4931-b9cf-779345109add None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:29.250584 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7db5a1-54e5-4931-b9cf-779345109add None admin] PATH_INFO: `/v3/users/swift` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:29.255749 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:29.262068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7db5a1-54e5-4931-b9cf-779345109add None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:29.263850 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7db5a1-54e5-4931-b9cf-779345109add None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:29.275447 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-eb7db5a1-54e5-4931-b9cf-779345109add None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:21:29.275447 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:29.275447 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:29.275447 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:29.275447 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:29.275447 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:29.275447 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:29.275447 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:29.275447 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:29.275447 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:29.275447 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:29.275447 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:29.275447 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:21:29.275447 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:21:29.275447 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:21:29.275447 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:21:29.276348 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:29.276348 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:29.276348 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:21:29.276348 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:29.276348 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:21:29.276348 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:29.276348 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:29.276348 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:29.276348 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:29.276348 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:29.276348 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:29.276348 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:29.276348 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:29.276348 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:29.276348 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:29.276348 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:29.276348 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:29.276348 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:29.277303 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:29.277303 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:29.277303 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:21:29.277303 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:21:29.277303 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:21:29.277303 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:21:29.277303 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:21:29.277303 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:21:29.277303 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:21:29.277303 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:29.279357 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 310/614] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:21:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:29.287355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b88b5-86df-42ca-9d0a-cbf9844f9504 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:29.413006 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:29.422824 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b88b5-86df-42ca-9d0a-cbf9844f9504 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:29.478964 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c413a-4246-472e-8967-00c1ab6bc73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:29.479626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c413a-4246-472e-8967-00c1ab6bc73b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:29.479910 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c413a-4246-472e-8967-00c1ab6bc73b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:29.480184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c413a-4246-472e-8967-00c1ab6bc73b None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:29.480925 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c413a-4246-472e-8967-00c1ab6bc73b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:29.481234 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c413a-4246-472e-8967-00c1ab6bc73b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:29.482771 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c413a-4246-472e-8967-00c1ab6bc73b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:29.505375 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 305/615] 10.222.0.18 () {56 vars in 1135 bytes} [Thu Sep 1 16:21:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:29.510616 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7db5a1-54e5-4931-b9cf-779345109add None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:29.594911 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270afe57-7d14-4c41-9813-f48c48faf258 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:29.595204 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270afe57-7d14-4c41-9813-f48c48faf258 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:29.595483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270afe57-7d14-4c41-9813-f48c48faf258 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:29.597492 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 306/616] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:21:29.603501 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830843a6-c9c8-4431-a420-d9c539757c24 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:29.603795 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830843a6-c9c8-4431-a420-d9c539757c24 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:29.604075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830843a6-c9c8-4431-a420-d9c539757c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:29.639572 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-830843a6-c9c8-4431-a420-d9c539757c24 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:29.640728 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-830843a6-c9c8-4431-a420-d9c539757c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:29.656436 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7db5a1-54e5-4931-b9cf-779345109add None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:29.706295 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f70853-6664-4a8d-a743-126c5ba49bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:29.707319 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f70853-6664-4a8d-a743-126c5ba49bfe None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:29.708825 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f70853-6664-4a8d-a743-126c5ba49bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:29.709740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f70853-6664-4a8d-a743-126c5ba49bfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:29.730328 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f70853-6664-4a8d-a743-126c5ba49bfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:29.733139 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f70853-6664-4a8d-a743-126c5ba49bfe None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:29.762263 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-42f70853-6664-4a8d-a743-126c5ba49bfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:29.762263 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:29.762263 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:29.762263 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:29.762263 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:29.762263 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:29.762263 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:29.762263 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:29.762263 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:29.762263 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:29.762263 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:29.762263 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:29.762263 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:29.762263 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:29.762263 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:29.762263 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:29.763313 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:29.763313 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:29.763313 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:29.763313 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:29.763313 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:29.763313 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:29.763313 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:29.763313 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:29.763313 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:29.763313 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:29.763313 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:29.763313 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:29.763313 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:29.763313 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:29.763313 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:29.763313 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:29.763313 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:29.764149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:29.764149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:29.764149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:29.764149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 311/617] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:29.764149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f70853-6664-4a8d-a743-126c5ba49bfe None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:29.779436 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 307/618] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:21:29.831850 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:29.901176 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f70853-6664-4a8d-a743-126c5ba49bfe None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:29.952208 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a6d24-e014-48b0-84d5-81a34435059f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:29.952827 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a6d24-e014-48b0-84d5-81a34435059f None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:29.953114 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a6d24-e014-48b0-84d5-81a34435059f None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:29.953394 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a6d24-e014-48b0-84d5-81a34435059f None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:29.954042 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2a6d24-e014-48b0-84d5-81a34435059f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:29.954382 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2a6d24-e014-48b0-84d5-81a34435059f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:29.955767 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2a6d24-e014-48b0-84d5-81a34435059f None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:29.974876 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 312/619] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:29.980867 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830843a6-c9c8-4431-a420-d9c539757c24 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:30.095886 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:30.122830 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830843a6-c9c8-4431-a420-d9c539757c24 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:30.136009 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d59376-5451-4c47-8b8b-628b33476574 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:30.136434 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d59376-5451-4c47-8b8b-628b33476574 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:30.136716 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d59376-5451-4c47-8b8b-628b33476574 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:30.138470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 313/620] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:30.146201 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35a2b4-4c10-4846-96f4-672a8dfe1ff9 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:30.146526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35a2b4-4c10-4846-96f4-672a8dfe1ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:30.146801 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35a2b4-4c10-4846-96f4-672a8dfe1ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:30.174258 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-9c35a2b4-4c10-4846-96f4-672a8dfe1ff9 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:30.175391 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-9c35a2b4-4c10-4846-96f4-672a8dfe1ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:30.180712 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d2e103-9712-4744-a233-64148cb46a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:30.181416 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d2e103-9712-4744-a233-64148cb46a8f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:30.181702 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d2e103-9712-4744-a233-64148cb46a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:30.181977 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d2e103-9712-4744-a233-64148cb46a8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:30.195486 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d2e103-9712-4744-a233-64148cb46a8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:30.196927 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d2e103-9712-4744-a233-64148cb46a8f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:30.213448 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-d3d2e103-9712-4744-a233-64148cb46a8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:30.213448 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:30.213448 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:30.213448 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:30.213448 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:30.213448 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:30.213448 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:30.213448 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:30.213448 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:30.213448 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:30.213448 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:30.213448 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:30.213448 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:21:30.213448 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:21:30.213448 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:21:30.214884 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:21:30.214884 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:30.214884 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:30.214884 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:30.214884 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:30.214884 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:30.214884 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:30.214884 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:30.214884 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:30.214884 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:30.214884 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:30.214884 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:30.214884 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:30.214884 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:30.214884 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:30.214884 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:30.214884 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:30.215728 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:21:30.215728 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:21:30.215728 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:21:30.215728 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:21:30.215728 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:21:30.215728 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:21:30.215728 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:30.215728 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:30.216771 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 308/621] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:30.220547 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d2e103-9712-4744-a233-64148cb46a8f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:30.310191 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 314/622] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:21:30.311345 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35a2b4-4c10-4846-96f4-672a8dfe1ff9 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:30.333183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d2e103-9712-4744-a233-64148cb46a8f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:30.371637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:30.379037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36747061-ede9-4690-8c5c-40fa6ca8e74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:30.379658 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36747061-ede9-4690-8c5c-40fa6ca8e74e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:30.379946 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36747061-ede9-4690-8c5c-40fa6ca8e74e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:30.380226 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36747061-ede9-4690-8c5c-40fa6ca8e74e None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:30.381204 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36747061-ede9-4690-8c5c-40fa6ca8e74e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:30.381521 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36747061-ede9-4690-8c5c-40fa6ca8e74e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:30.383399 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36747061-ede9-4690-8c5c-40fa6ca8e74e None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:30.394378 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:30.406504 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 309/623] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:21:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:30.406504 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e67da-5726-451c-8f93-07695ffb765f None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:30.406504 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e67da-5726-451c-8f93-07695ffb765f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:30.406504 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e67da-5726-451c-8f93-07695ffb765f None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:30.407652 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 310/624] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:30.408903 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e67da-5726-451c-8f93-07695ffb765f None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:30.453556 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35a2b4-4c10-4846-96f4-672a8dfe1ff9 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:30.498565 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54636e4e-5611-420e-8938-b93fd35cfe7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:30.499481 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54636e4e-5611-420e-8938-b93fd35cfe7a None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:30.499909 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54636e4e-5611-420e-8938-b93fd35cfe7a None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:30.500359 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54636e4e-5611-420e-8938-b93fd35cfe7a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:30.511911 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54636e4e-5611-420e-8938-b93fd35cfe7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:30.516830 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-54636e4e-5611-420e-8938-b93fd35cfe7a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:30.516830 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:30.516830 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:30.516830 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:30.516830 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:30.516830 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:30.516830 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:30.516830 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:30.516830 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:30.516830 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:30.516830 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:30.516830 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:30.516830 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:30.516830 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:30.516830 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:30.516830 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:30.517738 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:30.517738 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:30.517738 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:30.517738 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:30.517738 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:30.517738 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:30.517738 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:30.517738 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:30.517738 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:30.517738 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:30.517738 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:30.517738 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:30.517738 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:30.517738 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:30.517738 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:30.517738 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:30.518577 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:30.518577 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:30.518577 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:30.518577 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:30.518577 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:30.518577 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:30.518577 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:30.518577 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:30.518577 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:30.518577 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:30.518577 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:30.518577 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:30.518577 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 315/625] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 1 16:21:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:30.519742 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c632b2-f26b-44d9-a20e-6f632d00125f None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:30.519742 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c632b2-f26b-44d9-a20e-6f632d00125f None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:30.519892 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c632b2-f26b-44d9-a20e-6f632d00125f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:30.538473 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-f1c632b2-f26b-44d9-a20e-6f632d00125f None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:30.539692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-f1c632b2-f26b-44d9-a20e-6f632d00125f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:30.540606 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e67da-5726-451c-8f93-07695ffb765f None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:30.576429 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b595c4-ed2c-49e2-9468-98da6330e539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:30.577094 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b595c4-ed2c-49e2-9468-98da6330e539 None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:30.577377 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b595c4-ed2c-49e2-9468-98da6330e539 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:30.577654 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b595c4-ed2c-49e2-9468-98da6330e539 None admin] PATH_INFO: `/v3/projects/36ce80c4e0d64562a9ab3961be76fad9/users/8137f6b2ced040bd9752239c7e987f03/roles/b6727ed962fa41109b0a2d6851551f49` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:30.592589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b595c4-ed2c-49e2-9468-98da6330e539 None admin] RBAC: Authorizing `identity:create_grant(project_id=36ce80c4e0d64562a9ab3961be76fad9, user_id=8137f6b2ced040bd9752239c7e987f03, role_id=b6727ed962fa41109b0a2d6851551f49)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:30.594027 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b595c4-ed2c-49e2-9468-98da6330e539 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:30.607988 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 311/626] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 1 16:21:30 2022] PUT /identity/v3/projects/36ce80c4e0d64562a9ab3961be76fad9/users/8137f6b2ced040bd9752239c7e987f03/roles/b6727ed962fa41109b0a2d6851551f49 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:21:30.609323 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd764e4-08b0-44d4-bb22-ce419aafeeb0 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:30.609618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd764e4-08b0-44d4-bb22-ce419aafeeb0 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:30.609897 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd764e4-08b0-44d4-bb22-ce419aafeeb0 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:30.610128 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 316/627] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:21:30.614287 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 312/628] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:30.614429 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c632b2-f26b-44d9-a20e-6f632d00125f None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:30.614530 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd764e4-08b0-44d4-bb22-ce419aafeeb0 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:30.653746 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:30.688115 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:30.743655 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c632b2-f26b-44d9-a20e-6f632d00125f None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:30.750289 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd764e4-08b0-44d4-bb22-ce419aafeeb0 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:30.785357 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a767755-f4d7-453b-93c6-b3017b05c23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:30.785967 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a767755-f4d7-453b-93c6-b3017b05c23b None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:30.786312 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a767755-f4d7-453b-93c6-b3017b05c23b None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:30.786552 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a767755-f4d7-453b-93c6-b3017b05c23b None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:30.787217 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a767755-f4d7-453b-93c6-b3017b05c23b None admin] RBAC: Adding query filter params (name=service) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:30.787521 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a767755-f4d7-453b-93c6-b3017b05c23b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:30.788894 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a767755-f4d7-453b-93c6-b3017b05c23b None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:30.794984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8212df3-8a24-41ec-acf3-58e99d664c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:30.795867 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8212df3-8a24-41ec-acf3-58e99d664c23 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:30.796334 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8212df3-8a24-41ec-acf3-58e99d664c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:30.796805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8212df3-8a24-41ec-acf3-58e99d664c23 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:30.797251 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 317/629] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 1 16:21:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:30.798774 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691118b8-23c1-47c5-a8d9-9ea69fb933dc None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:30.799062 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691118b8-23c1-47c5-a8d9-9ea69fb933dc None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:30.799336 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691118b8-23c1-47c5-a8d9-9ea69fb933dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:30.809843 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8212df3-8a24-41ec-acf3-58e99d664c23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:30.815081 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-b8212df3-8a24-41ec-acf3-58e99d664c23 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:30.815081 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:30.815081 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:30.815081 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:30.815081 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:30.815081 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:30.815081 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:30.815081 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:30.815081 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:30.815081 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:30.815081 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:30.815081 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:30.815081 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:30.815081 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:30.815081 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:30.815081 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:30.816075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:30.816075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:30.816075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:30.816075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:30.816075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:30.816075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:30.816075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:30.816075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:30.816075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:30.816075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:30.816075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:30.816075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:30.816075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:30.816075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:30.816075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:30.816075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:30.816894 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:30.816894 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:30.816894 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:30.816894 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:30.816894 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:30.816894 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:30.816894 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:30.816894 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:30.816894 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:30.816894 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:30.816894 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:30.816894 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:30.817669 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-691118b8-23c1-47c5-a8d9-9ea69fb933dc None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:30.818955 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-691118b8-23c1-47c5-a8d9-9ea69fb933dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:30.819363 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 313/630] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 1 16:21:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:30.820420 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8212df3-8a24-41ec-acf3-58e99d664c23 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:30.896574 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 318/631] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:21:30.897646 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691118b8-23c1-47c5-a8d9-9ea69fb933dc None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:30.923398 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:30.941084 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8212df3-8a24-41ec-acf3-58e99d664c23 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:30.952105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:30.981074 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673fe57-212f-46f8-8f44-6c1f0857d4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:30.982765 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673fe57-212f-46f8-8f44-6c1f0857d4bf None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:30.983206 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673fe57-212f-46f8-8f44-6c1f0857d4bf None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:30.983636 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673fe57-212f-46f8-8f44-6c1f0857d4bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:30.996399 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4673fe57-212f-46f8-8f44-6c1f0857d4bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:30.998615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4673fe57-212f-46f8-8f44-6c1f0857d4bf None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:31.016080 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691118b8-23c1-47c5-a8d9-9ea69fb933dc None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:31.017149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-4673fe57-212f-46f8-8f44-6c1f0857d4bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:31.017149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:31.017149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:31.017149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:31.017149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:31.017149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:31.017149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:31.017149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:31.017149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:31.017149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:31.017149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:31.017149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:31.017149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:31.017149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:31.017149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:31.017149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:31.018265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:31.018265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:31.018265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:31.018265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:31.018265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:31.018265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:31.018265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:31.018265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:31.018265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:31.018265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:31.018265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:31.018265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:31.018265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:31.018265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:31.018265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:31.018265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:31.018265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:31.019360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:31.019360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:31.019360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:31.022030 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 314/632] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:31.023073 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673fe57-212f-46f8-8f44-6c1f0857d4bf None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:31.061156 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a521b7-c3f0-4ebb-8c02-71163dcccde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:31.062267 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a521b7-c3f0-4ebb-8c02-71163dcccde8 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:31.062932 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a521b7-c3f0-4ebb-8c02-71163dcccde8 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:31.063311 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a521b7-c3f0-4ebb-8c02-71163dcccde8 None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:31.064078 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a521b7-c3f0-4ebb-8c02-71163dcccde8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:31.064488 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a521b7-c3f0-4ebb-8c02-71163dcccde8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:31.066966 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a521b7-c3f0-4ebb-8c02-71163dcccde8 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:31.077456 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 319/633] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 1 16:21:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:31.079373 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a521b7-c3f0-4ebb-8c02-71163dcccde8 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:31.157547 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673fe57-212f-46f8-8f44-6c1f0857d4bf None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:31.187248 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:31.203846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a46970-a960-428d-bcba-3e2c3b33f699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:31.204482 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a46970-a960-428d-bcba-3e2c3b33f699 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:31.204769 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a46970-a960-428d-bcba-3e2c3b33f699 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:31.205045 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a46970-a960-428d-bcba-3e2c3b33f699 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:31.208276 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a521b7-c3f0-4ebb-8c02-71163dcccde8 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:31.217319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a46970-a960-428d-bcba-3e2c3b33f699 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:31.223300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-59a46970-a960-428d-bcba-3e2c3b33f699 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:31.223300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:31.223300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:31.223300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:31.223300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:31.223300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:31.223300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:31.223300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:31.223300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:31.223300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:31.223300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:31.223300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:31.223300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:31.223300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:31.223300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:31.223300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:31.224227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:31.224227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:31.224227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:31.224227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:31.224227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:31.224227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:31.224227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:31.224227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:31.224227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:31.224227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:31.224227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:31.224227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:31.224227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:31.224227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:31.224227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:31.224227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:31.225708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:31.225708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:31.225708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:31.225708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:31.225708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:31.225708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:31.225708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:31.225708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:31.225708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:31.225708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:31.225708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:31.225708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:31.225708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 315/634] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 1 16:21:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:31.225708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a46970-a960-428d-bcba-3e2c3b33f699 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:31.247517 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:31.252239 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3a3ac-4177-4cd7-8d9e-0656d3fb264a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:31.252755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3a3ac-4177-4cd7-8d9e-0656d3fb264a None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:31.253042 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3a3ac-4177-4cd7-8d9e-0656d3fb264a None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:31.253321 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3a3ac-4177-4cd7-8d9e-0656d3fb264a None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:31.253965 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3a3ac-4177-4cd7-8d9e-0656d3fb264a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:31.254312 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3a3ac-4177-4cd7-8d9e-0656d3fb264a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:31.255724 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3a3ac-4177-4cd7-8d9e-0656d3fb264a None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:31.268428 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 320/635] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:31.269053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3a3ac-4177-4cd7-8d9e-0656d3fb264a None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:31.338615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a46970-a960-428d-bcba-3e2c3b33f699 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:31.374078 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3a3ac-4177-4cd7-8d9e-0656d3fb264a None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:31.380639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2ec22-f4e3-4b61-a94b-bc447b5ad414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:31.381278 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2ec22-f4e3-4b61-a94b-bc447b5ad414 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:31.381566 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2ec22-f4e3-4b61-a94b-bc447b5ad414 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:31.381842 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2ec22-f4e3-4b61-a94b-bc447b5ad414 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:31.392927 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2ec22-f4e3-4b61-a94b-bc447b5ad414 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:31.394386 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2ec22-f4e3-4b61-a94b-bc447b5ad414 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:31.409513 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-4de2ec22-f4e3-4b61-a94b-bc447b5ad414 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:31.409513 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:31.409513 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:31.409513 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:31.409513 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:31.409513 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:31.409513 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:31.409513 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:31.409513 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:31.409513 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:31.409513 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:31.409513 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:31.409513 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:31.409513 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:31.409513 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:31.409513 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:31.410438 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:31.410438 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:31.410438 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:31.410438 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:31.410438 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:31.410438 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:31.410438 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:31.410438 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:31.410438 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:31.410438 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:31.410438 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:31.410438 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:31.410438 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:31.410438 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:31.410438 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:31.410438 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:31.410438 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:31.411306 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:31.411306 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:31.411306 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:31.411306 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 316/636] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:31.411523 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2ec22-f4e3-4b61-a94b-bc447b5ad414 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:31.414458 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca5779-c9bb-40e1-8d5b-6df4ad7c5bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:31.414999 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca5779-c9bb-40e1-8d5b-6df4ad7c5bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:31.415497 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca5779-c9bb-40e1-8d5b-6df4ad7c5bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:31.415609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca5779-c9bb-40e1-8d5b-6df4ad7c5bd0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:31.426077 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ca5779-c9bb-40e1-8d5b-6df4ad7c5bd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:31.430986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-00ca5779-c9bb-40e1-8d5b-6df4ad7c5bd0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:31.430986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:31.430986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:31.430986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:31.430986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:31.430986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:31.430986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:31.430986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:31.430986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:31.430986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:31.430986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:31.430986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:31.430986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:31.430986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:31.430986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:31.430986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:31.431916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:31.431916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:31.431916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:31.431916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:31.431916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:31.431916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:31.431916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:31.431916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:31.431916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:31.431916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:31.431916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:31.431916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:31.431916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:31.431916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:31.431916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:31.431916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:31.432779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:31.432779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:31.432779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:31.432779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:31.432779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:31.432779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:31.432779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:31.432779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:31.432779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:31.432779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:31.432779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:31.432779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:31.432779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 321/637] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 1 16:21:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:31.433551 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca5779-c9bb-40e1-8d5b-6df4ad7c5bd0 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:31.436595 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:31.499933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:31.532469 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2ec22-f4e3-4b61-a94b-bc447b5ad414 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:31.541184 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca5779-c9bb-40e1-8d5b-6df4ad7c5bd0 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:31.568360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20287d3-aeca-45cd-b20d-01670e3965d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:31.568975 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20287d3-aeca-45cd-b20d-01670e3965d9 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:31.569264 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20287d3-aeca-45cd-b20d-01670e3965d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:31.569544 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20287d3-aeca-45cd-b20d-01670e3965d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:31.570208 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20287d3-aeca-45cd-b20d-01670e3965d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:31.570547 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20287d3-aeca-45cd-b20d-01670e3965d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:31.571909 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20287d3-aeca-45cd-b20d-01670e3965d9 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:31.578836 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7fd141-ea3a-4a9b-8491-669048dc8b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:31.579674 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fd141-ea3a-4a9b-8491-669048dc8b76 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:31.580140 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fd141-ea3a-4a9b-8491-669048dc8b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:31.580490 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 317/638] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 1 16:21:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:31.580805 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fd141-ea3a-4a9b-8491-669048dc8b76 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:31.581291 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20287d3-aeca-45cd-b20d-01670e3965d9 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:31.591539 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7fd141-ea3a-4a9b-8491-669048dc8b76 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:31.593174 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7fd141-ea3a-4a9b-8491-669048dc8b76 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:31.605345 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-0f7fd141-ea3a-4a9b-8491-669048dc8b76 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 16:21:31.605345 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:31.605345 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:31.605345 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:31.605345 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:31.605345 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:31.605345 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:31.605345 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:31.605345 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:31.605345 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:31.605345 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:31.605345 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:31.605345 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:21:31.605345 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:21:31.605345 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:21:31.605345 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:21:31.606527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:31.606527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:31.606527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:21:31.606527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:31.606527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:21:31.606527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:31.606527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:31.606527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:31.606527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:31.606527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:31.606527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:31.606527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:31.606527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:31.606527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:31.606527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:31.606527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:31.606527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:31.606527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:31.607453 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:31.607453 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:31.607453 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:21:31.607453 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:21:31.607453 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:21:31.607453 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:21:31.607453 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:21:31.607453 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:21:31.607453 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 16:21:31.607453 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:31.607453 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 322/639] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:21:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:31.608024 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7fd141-ea3a-4a9b-8491-669048dc8b76 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:31.674327 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:31.692735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20287d3-aeca-45cd-b20d-01670e3965d9 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:31.719380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7fd141-ea3a-4a9b-8491-669048dc8b76 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:31.728795 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b22fa-39f7-4995-8bd2-981d8a8aa7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:31.729466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b22fa-39f7-4995-8bd2-981d8a8aa7b9 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:31.729760 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b22fa-39f7-4995-8bd2-981d8a8aa7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:31.730046 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b22fa-39f7-4995-8bd2-981d8a8aa7b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:31.730762 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b22fa-39f7-4995-8bd2-981d8a8aa7b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:31.731079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b22fa-39f7-4995-8bd2-981d8a8aa7b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:31.732485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b22fa-39f7-4995-8bd2-981d8a8aa7b9 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:31.734405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:31.747910 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 318/640] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:31.748748 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b22fa-39f7-4995-8bd2-981d8a8aa7b9 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:31.764398 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ea40e-8812-43c1-8dab-0505592dae00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:31.765385 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ea40e-8812-43c1-8dab-0505592dae00 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:31.765679 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ea40e-8812-43c1-8dab-0505592dae00 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:31.766105 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ea40e-8812-43c1-8dab-0505592dae00 None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:31.766811 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ea40e-8812-43c1-8dab-0505592dae00 None admin] RBAC: Adding query filter params (name=service) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:31.767216 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ea40e-8812-43c1-8dab-0505592dae00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:31.768566 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ea40e-8812-43c1-8dab-0505592dae00 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:31.777375 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 323/641] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 1 16:21:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:31.778264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ea40e-8812-43c1-8dab-0505592dae00 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:31.858017 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b22fa-39f7-4995-8bd2-981d8a8aa7b9 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:31.884396 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ea40e-8812-43c1-8dab-0505592dae00 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:31.895378 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8abd113-3b61-4b6c-a8c4-f88e32048f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:31.896011 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8abd113-3b61-4b6c-a8c4-f88e32048f9b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:31.896384 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8abd113-3b61-4b6c-a8c4-f88e32048f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:31.896606 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8abd113-3b61-4b6c-a8c4-f88e32048f9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:31.906850 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8abd113-3b61-4b6c-a8c4-f88e32048f9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:31.908275 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8abd113-3b61-4b6c-a8c4-f88e32048f9b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:31.909016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:31.921609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b75056-e580-4822-8239-32361bac2a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:31.922217 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b75056-e580-4822-8239-32361bac2a33 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:31.922535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b75056-e580-4822-8239-32361bac2a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:31.922814 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b75056-e580-4822-8239-32361bac2a33 None admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:31.923536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b75056-e580-4822-8239-32361bac2a33 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:31.923812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-d8abd113-3b61-4b6c-a8c4-f88e32048f9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:31.923812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:31.923812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:31.923812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:31.923812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:31.923812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:31.923812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:31.923812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:31.923812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:31.923812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:31.923812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:31.923812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:31.923812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:31.923812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:31.923812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:31.923812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:31.924762 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:31.924762 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:31.924762 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:31.924762 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:31.924762 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:31.924762 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:31.924762 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:31.924762 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:31.924762 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:31.924762 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:31.924762 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:31.924762 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:31.924762 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:31.924762 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:31.924762 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:31.924762 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:31.924762 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:31.925646 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:31.925646 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:31.925646 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:31.926334 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b75056-e580-4822-8239-32361bac2a33 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:31.926334 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b75056-e580-4822-8239-32361bac2a33 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:31.929682 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 319/642] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:31.930545 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8abd113-3b61-4b6c-a8c4-f88e32048f9b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:31.943851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 324/643] 10.222.0.18 () {56 vars in 1137 bytes} [Thu Sep 1 16:21:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:31.944475 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b75056-e580-4822-8239-32361bac2a33 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:31.986880 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:32.043614 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8abd113-3b61-4b6c-a8c4-f88e32048f9b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:32.052890 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b75056-e580-4822-8239-32361bac2a33 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:32.079009 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1df3ad-b284-4f11-80b5-1128481f5d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:32.079690 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1df3ad-b284-4f11-80b5-1128481f5d66 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:32.079980 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1df3ad-b284-4f11-80b5-1128481f5d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:32.080254 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1df3ad-b284-4f11-80b5-1128481f5d66 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:32.087886 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1df3ad-b284-4f11-80b5-1128481f5d66 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:32.088838 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbdc189-ffab-4c2b-af8a-d1991333c168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:32.089370 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1df3ad-b284-4f11-80b5-1128481f5d66 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:32.089473 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbdc189-ffab-4c2b-af8a-d1991333c168 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:32.089745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbdc189-ffab-4c2b-af8a-d1991333c168 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:32.090029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbdc189-ffab-4c2b-af8a-d1991333c168 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:32.100594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbdc189-ffab-4c2b-af8a-d1991333c168 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:32.101305 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-2e1df3ad-b284-4f11-80b5-1128481f5d66 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:21:32.101305 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:32.101305 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:32.101305 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:32.101305 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:32.101305 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:32.101305 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:32.101305 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:32.101305 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:32.101305 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:32.101305 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:32.101305 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:32.101305 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:21:32.101305 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:21:32.101305 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:21:32.101305 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:21:32.102404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:32.102404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:32.102404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:21:32.102404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:32.102404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:21:32.102404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:32.102404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:32.102404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:32.102404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:32.102404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:32.102404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:32.102404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:32.102404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:32.102404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:32.102404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:32.102404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:32.102404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:32.102404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:32.103693 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:32.103693 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:32.103693 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:21:32.103693 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:21:32.103693 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:21:32.103693 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:21:32.103693 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:21:32.103693 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:21:32.103693 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:21:32.103693 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:32.104318 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbdc189-ffab-4c2b-af8a-d1991333c168 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:32.106084 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 320/644] 10.222.0.18 () {56 vars in 1093 bytes} [Thu Sep 1 16:21:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:32.107013 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1df3ad-b284-4f11-80b5-1128481f5d66 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:32.119010 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-2cbdc189-ffab-4c2b-af8a-d1991333c168 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:32.119010 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:32.119010 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:32.119010 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:32.119010 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:32.119010 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:32.119010 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:32.119010 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:32.119010 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:32.119010 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:32.119010 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:32.119010 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:32.119010 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:32.119010 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:32.119010 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:32.119010 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:32.120755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:32.120755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:32.120755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:32.120755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:32.120755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:32.120755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:32.120755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:32.120755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:32.120755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:32.120755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:32.120755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:32.120755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:32.120755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:32.120755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:32.120755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:32.120755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:32.120755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:32.121896 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:32.121896 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:32.121896 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:32.121896 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 325/645] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:32.121896 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbdc189-ffab-4c2b-af8a-d1991333c168 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:32.163668 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:32.221366 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1df3ad-b284-4f11-80b5-1128481f5d66 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:32.228667 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:32.234229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbdc189-ffab-4c2b-af8a-d1991333c168 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:32.260254 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8047f-caa9-488c-820a-d523b1fabf1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:32.261143 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8047f-caa9-488c-820a-d523b1fabf1f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:32.261425 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8047f-caa9-488c-820a-d523b1fabf1f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:32.261703 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8047f-caa9-488c-820a-d523b1fabf1f None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:32.262382 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef8047f-caa9-488c-820a-d523b1fabf1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:32.262801 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef8047f-caa9-488c-820a-d523b1fabf1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:32.264063 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef8047f-caa9-488c-820a-d523b1fabf1f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:32.271284 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08771e04-ca1d-454e-98b6-f7420be03d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:32.272004 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08771e04-ca1d-454e-98b6-f7420be03d3d None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:32.272383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08771e04-ca1d-454e-98b6-f7420be03d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:32.272773 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08771e04-ca1d-454e-98b6-f7420be03d3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:32.276794 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 321/646] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:32.277627 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8047f-caa9-488c-820a-d523b1fabf1f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:32.350380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08771e04-ca1d-454e-98b6-f7420be03d3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:32.351872 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08771e04-ca1d-454e-98b6-f7420be03d3d None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:32.382465 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8047f-caa9-488c-820a-d523b1fabf1f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:32.386865 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:32.417675 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50fac33-e287-4ab1-a55a-003b656fd300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:32.418361 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fac33-e287-4ab1-a55a-003b656fd300 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:32.418651 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fac33-e287-4ab1-a55a-003b656fd300 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:32.418930 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fac33-e287-4ab1-a55a-003b656fd300 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:32.419663 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50fac33-e287-4ab1-a55a-003b656fd300 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:32.419973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50fac33-e287-4ab1-a55a-003b656fd300 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:32.421446 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50fac33-e287-4ab1-a55a-003b656fd300 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:32.438373 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 322/647] 10.222.0.18 () {56 vars in 1133 bytes} [Thu Sep 1 16:21:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:32.439196 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50fac33-e287-4ab1-a55a-003b656fd300 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:32.465230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 326/648] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:21:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:21:32.466204 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08771e04-ca1d-454e-98b6-f7420be03d3d None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:32.470547 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:32.551356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50fac33-e287-4ab1-a55a-003b656fd300 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:32.572582 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08771e04-ca1d-454e-98b6-f7420be03d3d None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:32.583902 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5fe22b-e498-46f7-8cf0-ee2c2e35b8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:32.584514 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fe22b-e498-46f7-8cf0-ee2c2e35b8ed None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:32.584796 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fe22b-e498-46f7-8cf0-ee2c2e35b8ed None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:32.585069 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fe22b-e498-46f7-8cf0-ee2c2e35b8ed None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:32.585711 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5fe22b-e498-46f7-8cf0-ee2c2e35b8ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:32.586015 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5fe22b-e498-46f7-8cf0-ee2c2e35b8ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:32.587401 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5fe22b-e498-46f7-8cf0-ee2c2e35b8ed None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:32.598826 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 323/649] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:32.599615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5fe22b-e498-46f7-8cf0-ee2c2e35b8ed None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:32.606573 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1057fcd7-edff-46ed-aa09-6d090104f0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:32.607147 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057fcd7-edff-46ed-aa09-6d090104f0b6 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:32.607433 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057fcd7-edff-46ed-aa09-6d090104f0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:32.607708 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057fcd7-edff-46ed-aa09-6d090104f0b6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:32.609600 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:32.616179 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1057fcd7-edff-46ed-aa09-6d090104f0b6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:32.617617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1057fcd7-edff-46ed-aa09-6d090104f0b6 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:32.629171 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-1057fcd7-edff-46ed-aa09-6d090104f0b6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:21:32.629171 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:32.629171 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:32.629171 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:32.629171 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:32.629171 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:32.629171 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:32.629171 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:32.629171 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:32.629171 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:32.629171 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:32.629171 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:32.629171 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:21:32.629171 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:21:32.629171 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:21:32.629171 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:21:32.630074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:32.630074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:32.630074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:21:32.630074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:32.630074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:21:32.630074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:32.630074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:32.630074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:32.630074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:32.630074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:32.630074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:32.630074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:32.630074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:32.630074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:32.630074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:32.630074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:32.630074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:32.630074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:32.631031 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:32.631031 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:32.631031 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:21:32.631031 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:21:32.631031 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:21:32.631031 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:21:32.631031 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:21:32.631031 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:21:32.631031 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:21:32.631031 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:32.631031 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 327/650] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:21:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:32.631031 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1057fcd7-edff-46ed-aa09-6d090104f0b6 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:32.685246 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:32.701958 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5fe22b-e498-46f7-8cf0-ee2c2e35b8ed None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:32.727970 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1057fcd7-edff-46ed-aa09-6d090104f0b6 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:32.738530 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3576950a-cc40-4777-b511-7508a8197aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:32.739140 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3576950a-cc40-4777-b511-7508a8197aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:32.739422 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3576950a-cc40-4777-b511-7508a8197aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:32.739695 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3576950a-cc40-4777-b511-7508a8197aa9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:32.761387 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f03c4-d8fd-4a98-a75e-9641985c73d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:32.762007 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f03c4-d8fd-4a98-a75e-9641985c73d9 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:32.762323 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f03c4-d8fd-4a98-a75e-9641985c73d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:32.762608 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f03c4-d8fd-4a98-a75e-9641985c73d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:32.772840 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05f03c4-d8fd-4a98-a75e-9641985c73d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:32.774310 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05f03c4-d8fd-4a98-a75e-9641985c73d9 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:32.788482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-d05f03c4-d8fd-4a98-a75e-9641985c73d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:32.788482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:32.788482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:32.788482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:32.788482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:32.788482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:32.788482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:32.788482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:32.788482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:32.788482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:32.788482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:32.788482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:32.788482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:32.788482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:32.788482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:32.788482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:32.789441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:32.789441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:32.789441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:32.789441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:32.789441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:32.789441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:32.789441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:32.789441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:32.789441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:32.789441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:32.789441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:32.789441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:32.789441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:32.789441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:32.789441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:32.789441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:32.789441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:32.790813 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:32.790813 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:32.790813 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:32.790813 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 328/651] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:32.790813 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9aed03-21e3-4879-9cd7-98a061b44bc4 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:32.791170 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9aed03-21e3-4879-9cd7-98a061b44bc4 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:32.791254 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9aed03-21e3-4879-9cd7-98a061b44bc4 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:32.792403 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 329/652] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:32.793088 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9aed03-21e3-4879-9cd7-98a061b44bc4 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:32.810998 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3576950a-cc40-4777-b511-7508a8197aa9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:32.812453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3576950a-cc40-4777-b511-7508a8197aa9 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:32.840397 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:32.894978 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9aed03-21e3-4879-9cd7-98a061b44bc4 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:32.897115 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:32.914013 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 324/653] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:21:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:21:32.914869 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3576950a-cc40-4777-b511-7508a8197aa9 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:32.932014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb28ad-6a12-4d3e-8cf5-902e97bf33cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:32.932673 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb28ad-6a12-4d3e-8cf5-902e97bf33cc None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:32.932958 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb28ad-6a12-4d3e-8cf5-902e97bf33cc None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:32.933230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb28ad-6a12-4d3e-8cf5-902e97bf33cc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:32.940166 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb28ad-6a12-4d3e-8cf5-902e97bf33cc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:32.941609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb28ad-6a12-4d3e-8cf5-902e97bf33cc None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:32.953509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-72bb28ad-6a12-4d3e-8cf5-902e97bf33cc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 16:21:32.953509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:32.953509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:32.953509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:32.953509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:32.953509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:32.953509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:32.953509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:32.953509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:32.953509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:32.953509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:32.953509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:32.953509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:21:32.953509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:21:32.953509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:21:32.953509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:21:32.954554 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:32.954554 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:32.954554 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:21:32.954554 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:32.954554 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:21:32.954554 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:32.954554 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:32.954554 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:32.954554 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:32.954554 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:32.954554 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:32.954554 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:32.954554 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:32.954554 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:32.954554 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:32.954554 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:32.954554 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:32.954554 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:32.955503 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:32.955503 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:32.955503 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:21:32.955503 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:21:32.955503 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:21:32.955503 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:21:32.955503 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:21:32.955503 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:21:32.955503 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 16:21:32.955503 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:32.955503 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 330/654] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 1 16:21:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:32.955503 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb28ad-6a12-4d3e-8cf5-902e97bf33cc None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:33.016198 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3576950a-cc40-4777-b511-7508a8197aa9 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:33.049112 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2912e-8051-4c18-a5de-0e2d7b0605af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:33.049714 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2912e-8051-4c18-a5de-0e2d7b0605af None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:33.049996 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2912e-8051-4c18-a5de-0e2d7b0605af None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:33.050298 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2912e-8051-4c18-a5de-0e2d7b0605af None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:33.051019 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac2912e-8051-4c18-a5de-0e2d7b0605af None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:33.051326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac2912e-8051-4c18-a5de-0e2d7b0605af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:33.052735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac2912e-8051-4c18-a5de-0e2d7b0605af None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:33.053026 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb28ad-6a12-4d3e-8cf5-902e97bf33cc None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:33.060387 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:33.070644 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 325/655] 10.222.0.18 () {56 vars in 1137 bytes} [Thu Sep 1 16:21:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:33.071836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767af219-93b3-467d-8018-8c9918de7a29 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:33.072127 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767af219-93b3-467d-8018-8c9918de7a29 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:33.072406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767af219-93b3-467d-8018-8c9918de7a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:33.087544 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8365c43-035b-4316-b9e7-14ac2d62f8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:33.088149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8365c43-035b-4316-b9e7-14ac2d62f8c0 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:33.088431 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8365c43-035b-4316-b9e7-14ac2d62f8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:33.088732 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8365c43-035b-4316-b9e7-14ac2d62f8c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:33.089365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8365c43-035b-4316-b9e7-14ac2d62f8c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:33.089670 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8365c43-035b-4316-b9e7-14ac2d62f8c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:33.090099 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-767af219-93b3-467d-8018-8c9918de7a29 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:33.091050 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8365c43-035b-4316-b9e7-14ac2d62f8c0 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:33.091334 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-767af219-93b3-467d-8018-8c9918de7a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:33.102102 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 331/656] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:33.102903 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8365c43-035b-4316-b9e7-14ac2d62f8c0 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:33.126389 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:33.158708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 326/657] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:21:33.159534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767af219-93b3-467d-8018-8c9918de7a29 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:33.202478 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8365c43-035b-4316-b9e7-14ac2d62f8c0 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:33.234621 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841915a-dc8d-48c8-a388-a3a2e3baa13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:33.235243 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841915a-dc8d-48c8-a388-a3a2e3baa13e None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:33.235528 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841915a-dc8d-48c8-a388-a3a2e3baa13e None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:33.235801 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841915a-dc8d-48c8-a388-a3a2e3baa13e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:33.246868 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c841915a-dc8d-48c8-a388-a3a2e3baa13e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:33.248310 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c841915a-dc8d-48c8-a388-a3a2e3baa13e None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:33.262939 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-c841915a-dc8d-48c8-a388-a3a2e3baa13e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:33.262939 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:33.262939 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:33.262939 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:33.262939 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:33.262939 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:33.262939 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:33.262939 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:33.262939 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:33.262939 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:33.262939 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:33.262939 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:33.262939 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:33.262939 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:33.262939 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:33.262939 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:33.264037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:33.264037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:33.264037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:33.264037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:33.264037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:33.264037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:33.264037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:33.264037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:33.264037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:33.264037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:33.264037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:33.264037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:33.264037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:33.264037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:33.264037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:33.264037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:33.264037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:33.265162 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:33.265162 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:33.265162 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:33.265340 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767af219-93b3-467d-8018-8c9918de7a29 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:33.265413 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 332/658] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:33.265413 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841915a-dc8d-48c8-a388-a3a2e3baa13e None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:33.287267 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:33.301735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee0b0c-6b3d-4f17-966f-cfdaea4693af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:33.302385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee0b0c-6b3d-4f17-966f-cfdaea4693af None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:33.302671 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee0b0c-6b3d-4f17-966f-cfdaea4693af None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:33.302951 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee0b0c-6b3d-4f17-966f-cfdaea4693af None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:33.303680 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee0b0c-6b3d-4f17-966f-cfdaea4693af None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:33.303989 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee0b0c-6b3d-4f17-966f-cfdaea4693af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:33.305402 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee0b0c-6b3d-4f17-966f-cfdaea4693af None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:33.328136 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 327/659] 10.222.0.18 () {56 vars in 1139 bytes} [Thu Sep 1 16:21:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:33.328136 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee0b0c-6b3d-4f17-966f-cfdaea4693af None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:33.357444 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:33.387909 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841915a-dc8d-48c8-a388-a3a2e3baa13e None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:33.426174 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888fb6f7-be69-4fbf-92a0-df638913d33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:33.426748 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888fb6f7-be69-4fbf-92a0-df638913d33e None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:33.427032 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888fb6f7-be69-4fbf-92a0-df638913d33e None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:33.427304 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888fb6f7-be69-4fbf-92a0-df638913d33e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:33.430148 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee0b0c-6b3d-4f17-966f-cfdaea4693af None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:33.464494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346a4962-a6a8-4f43-93cf-7248b4d2485d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:33.465116 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a4962-a6a8-4f43-93cf-7248b4d2485d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:33.465396 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a4962-a6a8-4f43-93cf-7248b4d2485d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:33.465672 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a4962-a6a8-4f43-93cf-7248b4d2485d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:33.472415 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346a4962-a6a8-4f43-93cf-7248b4d2485d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:33.473828 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346a4962-a6a8-4f43-93cf-7248b4d2485d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:33.485242 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-346a4962-a6a8-4f43-93cf-7248b4d2485d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:33.485242 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:33.485242 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:33.485242 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:33.485242 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:33.485242 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:33.485242 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:33.485242 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:33.485242 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:33.485242 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:33.485242 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:33.485242 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:33.485242 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:21:33.485242 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:21:33.485242 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:21:33.486326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:21:33.486326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:33.486326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:33.486326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:33.486326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:33.486326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:33.486326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:33.486326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:33.486326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:33.486326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:33.486326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:33.486326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:33.486326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:33.486326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:33.486326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:33.486326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:33.486326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:33.487355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:21:33.487355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:21:33.487355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:21:33.487355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:21:33.487355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:21:33.487355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:21:33.487355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:33.487355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:33.487355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 328/660] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:33.487355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346a4962-a6a8-4f43-93cf-7248b4d2485d None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:33.497212 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888fb6f7-be69-4fbf-92a0-df638913d33e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:33.498700 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888fb6f7-be69-4fbf-92a0-df638913d33e None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:33.516245 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:33.575806 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:33.585290 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346a4962-a6a8-4f43-93cf-7248b4d2485d None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:33.595574 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 333/661] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:21:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:21:33.596397 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888fb6f7-be69-4fbf-92a0-df638913d33e None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:33.621129 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a22e5-04cd-467d-8b0c-a9e4884a2540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:33.621740 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a22e5-04cd-467d-8b0c-a9e4884a2540 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:33.622019 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a22e5-04cd-467d-8b0c-a9e4884a2540 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:33.622315 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a22e5-04cd-467d-8b0c-a9e4884a2540 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:33.622950 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a22e5-04cd-467d-8b0c-a9e4884a2540 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:33.623251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a22e5-04cd-467d-8b0c-a9e4884a2540 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:33.624605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a22e5-04cd-467d-8b0c-a9e4884a2540 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:33.636009 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 329/662] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:33.636782 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a22e5-04cd-467d-8b0c-a9e4884a2540 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:33.697668 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888fb6f7-be69-4fbf-92a0-df638913d33e None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:33.714458 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:33.731276 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97790006-88a2-4932-adc0-061ce9a945f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:33.731872 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97790006-88a2-4932-adc0-061ce9a945f4 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:33.732149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97790006-88a2-4932-adc0-061ce9a945f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:33.732418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97790006-88a2-4932-adc0-061ce9a945f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:33.732744 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a22e5-04cd-467d-8b0c-a9e4884a2540 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:33.767501 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca5afc-9e36-4032-94ba-22eedd430e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:33.768130 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca5afc-9e36-4032-94ba-22eedd430e01 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:33.768419 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca5afc-9e36-4032-94ba-22eedd430e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:33.768726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca5afc-9e36-4032-94ba-22eedd430e01 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:33.780888 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ca5afc-9e36-4032-94ba-22eedd430e01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:33.786128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-98ca5afc-9e36-4032-94ba-22eedd430e01 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:33.786128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:33.786128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:33.786128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:33.786128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:33.786128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:33.786128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:33.786128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:33.786128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:33.786128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:33.786128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:33.786128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:33.786128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:33.786128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:33.786128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:33.786128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:33.787072 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:33.787072 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:33.787072 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:33.787072 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:33.787072 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:33.787072 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:33.787072 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:33.787072 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:33.787072 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:33.787072 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:33.787072 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:33.787072 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:33.787072 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:33.787072 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:33.787072 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:33.787072 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:33.788037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:33.788037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:33.788037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:33.788037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:33.788037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:33.788037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:33.788037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:33.788037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:33.788037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:33.788037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:33.788037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:33.788037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:33.788037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 330/663] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 1 16:21:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:33.788924 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca5afc-9e36-4032-94ba-22eedd430e01 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:33.800546 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97790006-88a2-4932-adc0-061ce9a945f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:33.801371 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:33.802028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97790006-88a2-4932-adc0-061ce9a945f4 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:33.887095 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca5afc-9e36-4032-94ba-22eedd430e01 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:33.896770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 334/664] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:21:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:21:33.897587 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97790006-88a2-4932-adc0-061ce9a945f4 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:33.917658 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:33.922614 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45871923-38e9-4c22-8d3d-c6df77d284e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:33.923216 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45871923-38e9-4c22-8d3d-c6df77d284e1 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:33.923495 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45871923-38e9-4c22-8d3d-c6df77d284e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:33.923763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45871923-38e9-4c22-8d3d-c6df77d284e1 None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:33.924448 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45871923-38e9-4c22-8d3d-c6df77d284e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:33.924749 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45871923-38e9-4c22-8d3d-c6df77d284e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:33.926163 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45871923-38e9-4c22-8d3d-c6df77d284e1 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:33.937288 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 331/665] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:21:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:33.938061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45871923-38e9-4c22-8d3d-c6df77d284e1 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:33.997411 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97790006-88a2-4932-adc0-061ce9a945f4 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:34.007369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:34.030651 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e03c6-a015-4951-a5f7-00918e1745b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:34.031275 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e03c6-a015-4951-a5f7-00918e1745b8 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:34.031558 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e03c6-a015-4951-a5f7-00918e1745b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:34.031833 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e03c6-a015-4951-a5f7-00918e1745b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:34.038280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45871923-38e9-4c22-8d3d-c6df77d284e1 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:34.041511 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81e03c6-a015-4951-a5f7-00918e1745b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:34.043002 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81e03c6-a015-4951-a5f7-00918e1745b8 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:34.058099 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-e81e03c6-a015-4951-a5f7-00918e1745b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:34.058099 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:34.058099 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:34.058099 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:34.058099 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:34.058099 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:34.058099 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:34.058099 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:34.058099 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:34.058099 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:34.058099 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:34.058099 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:34.058099 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:34.058099 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:34.058099 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:34.058099 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:34.059068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:34.059068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:34.059068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:34.059068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:34.059068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:34.059068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:34.059068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:34.059068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:34.059068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:34.059068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:34.059068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:34.059068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:34.059068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:34.059068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:34.059068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:34.059068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:34.059068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:34.059937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:34.059937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:34.059937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:34.059937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 335/666] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:34.060163 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e03c6-a015-4951-a5f7-00918e1745b8 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:34.076970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9541b4b1-ffca-421f-817c-de101d2c1cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:34.077608 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541b4b1-ffca-421f-817c-de101d2c1cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:34.077896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541b4b1-ffca-421f-817c-de101d2c1cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:34.078174 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541b4b1-ffca-421f-817c-de101d2c1cd8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:34.084858 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9541b4b1-ffca-421f-817c-de101d2c1cd8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:34.086300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9541b4b1-ffca-421f-817c-de101d2c1cd8 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:34.097268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-9541b4b1-ffca-421f-817c-de101d2c1cd8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:34.097268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:34.097268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:34.097268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:34.097268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:34.097268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:34.097268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:34.097268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:34.097268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:34.097268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:34.097268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:34.097268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:34.097268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:21:34.097268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:21:34.097268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:21:34.098112 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:21:34.098112 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:34.098112 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:34.098112 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:34.098112 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:34.098112 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:34.098112 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:34.098112 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:34.098112 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:34.098112 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:34.098112 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:34.098112 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:34.098112 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:34.098112 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:34.098112 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:34.098112 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:34.098112 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:34.099029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:21:34.099029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:21:34.099029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:21:34.099029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:21:34.099029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:21:34.099029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:21:34.099029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:34.099029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:34.102559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 332/667] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:34.103444 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9541b4b1-ffca-421f-817c-de101d2c1cd8 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:34.131797 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:34.163238 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e03c6-a015-4951-a5f7-00918e1745b8 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:34.196595 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db555e9-a50b-4f77-8dd8-8b116794b73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:34.197196 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db555e9-a50b-4f77-8dd8-8b116794b73c None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:34.197474 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db555e9-a50b-4f77-8dd8-8b116794b73c None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:34.197744 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db555e9-a50b-4f77-8dd8-8b116794b73c None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:34.198448 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db555e9-a50b-4f77-8dd8-8b116794b73c None admin] RBAC: Adding query filter params (name=service) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:34.198751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db555e9-a50b-4f77-8dd8-8b116794b73c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:34.200104 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db555e9-a50b-4f77-8dd8-8b116794b73c None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:34.202449 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9541b4b1-ffca-421f-817c-de101d2c1cd8 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:34.208109 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 336/668] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 1 16:21:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:34.208876 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db555e9-a50b-4f77-8dd8-8b116794b73c None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:34.231209 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:34.236506 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6d3c4-2c19-4706-8abc-b572e2b515a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:34.237140 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6d3c4-2c19-4706-8abc-b572e2b515a1 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:34.237425 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6d3c4-2c19-4706-8abc-b572e2b515a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:34.237699 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6d3c4-2c19-4706-8abc-b572e2b515a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:34.249702 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6d3c4-2c19-4706-8abc-b572e2b515a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:34.251106 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6d3c4-2c19-4706-8abc-b572e2b515a1 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:34.264946 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-57a6d3c4-2c19-4706-8abc-b572e2b515a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:34.264946 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:34.264946 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:34.264946 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:34.264946 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:34.264946 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:34.264946 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:34.264946 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:34.264946 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:34.264946 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:34.264946 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:34.264946 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:34.264946 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:34.264946 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:34.264946 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:34.264946 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:34.265906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:34.265906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:34.265906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:34.265906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:34.265906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:34.265906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:34.265906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:34.265906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:34.265906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:34.265906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:34.265906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:34.265906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:34.265906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:34.265906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:34.265906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:34.265906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:34.265906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:34.266978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:34.266978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:34.266978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:34.266978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 333/669] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:34.266978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6d3c4-2c19-4706-8abc-b572e2b515a1 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:34.314423 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db555e9-a50b-4f77-8dd8-8b116794b73c None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:34.342073 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:34.347449 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686daaa-20ad-4988-98f2-05741dbdc283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:34.348059 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686daaa-20ad-4988-98f2-05741dbdc283 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:34.348341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686daaa-20ad-4988-98f2-05741dbdc283 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:34.348616 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686daaa-20ad-4988-98f2-05741dbdc283 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:34.349337 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686daaa-20ad-4988-98f2-05741dbdc283 None admin] RBAC: Adding query filter params (role.id=b6727ed962fa41109b0a2d6851551f49, scope.project.id=36ce80c4e0d64562a9ab3961be76fad9, user.id=b09f6d550c9142bf956b225bf5172e42) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:34.349648 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686daaa-20ad-4988-98f2-05741dbdc283 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:34.351233 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686daaa-20ad-4988-98f2-05741dbdc283 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:34.357491 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 337/670] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:21:34 2022] GET /identity/v3/role_assignments?user.id=b09f6d550c9142bf956b225bf5172e42&scope.project.id=36ce80c4e0d64562a9ab3961be76fad9&role.id=b6727ed962fa41109b0a2d6851551f49 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:34.358285 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686daaa-20ad-4988-98f2-05741dbdc283 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:34.374307 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6d3c4-2c19-4706-8abc-b572e2b515a1 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:34.413797 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b61d1ee-cffd-461d-9d28-faf1fce01385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:34.414455 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61d1ee-cffd-461d-9d28-faf1fce01385 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:34.414739 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61d1ee-cffd-461d-9d28-faf1fce01385 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:34.415011 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61d1ee-cffd-461d-9d28-faf1fce01385 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:34.415670 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b61d1ee-cffd-461d-9d28-faf1fce01385 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:34.415973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b61d1ee-cffd-461d-9d28-faf1fce01385 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:34.417895 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b61d1ee-cffd-461d-9d28-faf1fce01385 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:34.436635 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 334/671] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:34.436635 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b61d1ee-cffd-461d-9d28-faf1fce01385 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:34.481701 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:34.492552 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686daaa-20ad-4988-98f2-05741dbdc283 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:34.532575 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617b813-50ac-4796-8f17-c3efa73557cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:34.533221 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617b813-50ac-4796-8f17-c3efa73557cc None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:34.533505 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617b813-50ac-4796-8f17-c3efa73557cc None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:34.533780 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617b813-50ac-4796-8f17-c3efa73557cc None admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:34.534757 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a617b813-50ac-4796-8f17-c3efa73557cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:34.534861 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a617b813-50ac-4796-8f17-c3efa73557cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:34.536244 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a617b813-50ac-4796-8f17-c3efa73557cc None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:34.548777 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b61d1ee-cffd-461d-9d28-faf1fce01385 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:34.552148 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 338/672] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:21:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:34.552804 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617b813-50ac-4796-8f17-c3efa73557cc None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:34.590644 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83adf83-9ad9-4f8d-8f80-f8a86f403ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:34.591400 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83adf83-9ad9-4f8d-8f80-f8a86f403ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:34.591689 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83adf83-9ad9-4f8d-8f80-f8a86f403ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:34.592074 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83adf83-9ad9-4f8d-8f80-f8a86f403ae0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:34.602762 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83adf83-9ad9-4f8d-8f80-f8a86f403ae0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:34.605597 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83adf83-9ad9-4f8d-8f80-f8a86f403ae0 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:34.614551 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:34.623638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-b83adf83-9ad9-4f8d-8f80-f8a86f403ae0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:34.623638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:34.623638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:34.623638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:34.623638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:34.623638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:34.623638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:34.623638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:34.623638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:34.623638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:34.623638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:34.623638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:34.623638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:34.623638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:34.623638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:34.623638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:34.624601 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:34.624601 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:34.624601 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:34.624601 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:34.624601 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:34.624601 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:34.624601 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:34.624601 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:34.624601 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:34.624601 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:34.624601 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:34.624601 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:34.624601 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:34.624601 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:34.624601 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:34.624601 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:34.624601 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:34.625474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:34.625474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:34.625474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:34.625474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 335/673] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:34.626225 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83adf83-9ad9-4f8d-8f80-f8a86f403ae0 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:34.667842 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617b813-50ac-4796-8f17-c3efa73557cc None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:34.703782 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb3577e-a9b7-4b06-8419-47e8103055e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:34.704401 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3577e-a9b7-4b06-8419-47e8103055e9 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:34.704774 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3577e-a9b7-4b06-8419-47e8103055e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:34.705051 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3577e-a9b7-4b06-8419-47e8103055e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:34.705690 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb3577e-a9b7-4b06-8419-47e8103055e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:34.705994 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb3577e-a9b7-4b06-8419-47e8103055e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:34.707530 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb3577e-a9b7-4b06-8419-47e8103055e9 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:34.725165 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 339/674] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:34.725165 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb3577e-a9b7-4b06-8419-47e8103055e9 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:34.730803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:34.738168 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83adf83-9ad9-4f8d-8f80-f8a86f403ae0 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:34.777674 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572800d-9ba1-44b4-b614-91cba2ec2c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:34.778328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572800d-9ba1-44b4-b614-91cba2ec2c39 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:34.778616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572800d-9ba1-44b4-b614-91cba2ec2c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:34.778887 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572800d-9ba1-44b4-b614-91cba2ec2c39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:34.785591 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9572800d-9ba1-44b4-b614-91cba2ec2c39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:34.786987 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9572800d-9ba1-44b4-b614-91cba2ec2c39 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:34.798497 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-9572800d-9ba1-44b4-b614-91cba2ec2c39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:34.798497 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:34.798497 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:34.798497 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:34.798497 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:34.798497 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:34.798497 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:34.798497 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:34.798497 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:34.798497 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:34.798497 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:34.798497 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:34.798497 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:21:34.798497 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:21:34.798497 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:21:34.799344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:21:34.799344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:34.799344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:34.799344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:34.799344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:34.799344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:34.799344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:34.799344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:34.799344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:34.799344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:34.799344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:34.799344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:34.799344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:34.799344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:34.799344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:34.799344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:34.799344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:34.800222 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:21:34.800222 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:21:34.800222 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:21:34.800222 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:21:34.800222 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:21:34.800222 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:21:34.800222 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:34.800222 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:34.804233 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 336/675] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:34.805078 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572800d-9ba1-44b4-b614-91cba2ec2c39 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:34.833264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb3577e-a9b7-4b06-8419-47e8103055e9 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:34.851420 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:34.873154 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1624614-fdea-4524-b08b-c2c2cc1f0d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:34.873766 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1624614-fdea-4524-b08b-c2c2cc1f0d25 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:34.874044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1624614-fdea-4524-b08b-c2c2cc1f0d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:34.874352 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1624614-fdea-4524-b08b-c2c2cc1f0d25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:34.875085 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1624614-fdea-4524-b08b-c2c2cc1f0d25 None admin] RBAC: Adding query filter params (role.id=b6727ed962fa41109b0a2d6851551f49, scope.project.id=36ce80c4e0d64562a9ab3961be76fad9, user.id=278e6d6aed414c758fe3e9f4df6bf81e) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:34.875394 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1624614-fdea-4524-b08b-c2c2cc1f0d25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:34.876857 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1624614-fdea-4524-b08b-c2c2cc1f0d25 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:34.886556 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 340/676] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:21:34 2022] GET /identity/v3/role_assignments?user.id=278e6d6aed414c758fe3e9f4df6bf81e&scope.project.id=36ce80c4e0d64562a9ab3961be76fad9&role.id=b6727ed962fa41109b0a2d6851551f49 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:34.887361 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1624614-fdea-4524-b08b-c2c2cc1f0d25 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:34.918947 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572800d-9ba1-44b4-b614-91cba2ec2c39 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:34.960053 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea236d-9fca-4fec-ba5c-17649407a701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:34.961124 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea236d-9fca-4fec-ba5c-17649407a701 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:34.961625 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea236d-9fca-4fec-ba5c-17649407a701 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:34.962226 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea236d-9fca-4fec-ba5c-17649407a701 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:34.963256 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea236d-9fca-4fec-ba5c-17649407a701 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:34.964378 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea236d-9fca-4fec-ba5c-17649407a701 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:34.965994 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea236d-9fca-4fec-ba5c-17649407a701 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:34.981495 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 337/677] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:34.982734 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea236d-9fca-4fec-ba5c-17649407a701 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:35.000918 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:35.011766 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1624614-fdea-4524-b08b-c2c2cc1f0d25 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:35.049012 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0a4049-e6f9-4d1e-ac7d-8d33b86bdbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:35.049572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0a4049-e6f9-4d1e-ac7d-8d33b86bdbff None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:35.049851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0a4049-e6f9-4d1e-ac7d-8d33b86bdbff None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:35.050127 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0a4049-e6f9-4d1e-ac7d-8d33b86bdbff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:35.057081 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0a4049-e6f9-4d1e-ac7d-8d33b86bdbff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:35.058588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0a4049-e6f9-4d1e-ac7d-8d33b86bdbff None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:35.071461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-6c0a4049-e6f9-4d1e-ac7d-8d33b86bdbff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:35.071461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:35.071461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:35.071461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:35.071461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:35.071461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:35.071461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:35.071461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:35.071461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:35.071461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:35.071461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:35.071461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:35.071461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:21:35.071461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:21:35.071461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:21:35.072365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:21:35.072365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:35.072365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:35.072365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:35.072365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:35.072365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:35.072365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:35.072365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:35.072365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:35.072365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:35.072365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:35.072365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:35.072365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:35.072365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:35.072365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:35.072365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:35.072365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:35.073768 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:21:35.073768 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:21:35.073768 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:21:35.073768 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:21:35.073768 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:21:35.073768 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:21:35.073768 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:35.073768 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:35.073768 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 341/678] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:35.074296 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0a4049-e6f9-4d1e-ac7d-8d33b86bdbff None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:35.107154 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea236d-9fca-4fec-ba5c-17649407a701 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:35.130392 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:35.149692 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b58201-d0cf-4c0d-9fdb-e22e37d66859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:35.150531 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b58201-d0cf-4c0d-9fdb-e22e37d66859 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:35.150883 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b58201-d0cf-4c0d-9fdb-e22e37d66859 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:35.151803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b58201-d0cf-4c0d-9fdb-e22e37d66859 None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:35.152739 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b58201-d0cf-4c0d-9fdb-e22e37d66859 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:35.153435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b58201-d0cf-4c0d-9fdb-e22e37d66859 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:35.155208 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b58201-d0cf-4c0d-9fdb-e22e37d66859 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:35.169193 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 338/679] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:21:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:35.170168 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b58201-d0cf-4c0d-9fdb-e22e37d66859 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:35.202943 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0a4049-e6f9-4d1e-ac7d-8d33b86bdbff None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:35.252399 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:35.253323 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e7d14-e124-4017-ae53-72886249963f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:35.254037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e7d14-e124-4017-ae53-72886249963f None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:35.254418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e7d14-e124-4017-ae53-72886249963f None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:35.254699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e7d14-e124-4017-ae53-72886249963f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:35.266284 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72e7d14-e124-4017-ae53-72886249963f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:35.268142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72e7d14-e124-4017-ae53-72886249963f None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:35.282734 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b58201-d0cf-4c0d-9fdb-e22e37d66859 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:35.283480 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-e72e7d14-e124-4017-ae53-72886249963f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:21:35.283480 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:35.283480 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:35.283480 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:35.283480 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:35.283480 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:35.283480 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:35.283480 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:35.283480 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:35.283480 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:35.283480 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:35.283480 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:35.283480 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:21:35.283480 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:21:35.283480 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:21:35.283480 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:21:35.284369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:35.284369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:35.284369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:21:35.284369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:35.284369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:21:35.284369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:35.284369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:35.284369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:35.284369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:35.284369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:35.284369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:35.284369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:35.284369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:35.284369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:35.284369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:35.284369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:35.284369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:35.284369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:35.285282 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:35.285282 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:35.285282 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:21:35.285282 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:21:35.285282 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:21:35.285282 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:21:35.285282 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:21:35.285282 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:21:35.285282 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:21:35.285282 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:35.285282 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 342/680] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:21:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:35.285872 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e7d14-e124-4017-ae53-72886249963f None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:35.332841 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec510f-0edb-4e00-a811-ce77f0bc9e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:35.333427 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec510f-0edb-4e00-a811-ce77f0bc9e9b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:35.333711 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec510f-0edb-4e00-a811-ce77f0bc9e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:35.333985 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec510f-0edb-4e00-a811-ce77f0bc9e9b None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:35.335257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aec510f-0edb-4e00-a811-ce77f0bc9e9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:35.335570 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aec510f-0edb-4e00-a811-ce77f0bc9e9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:35.337441 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aec510f-0edb-4e00-a811-ce77f0bc9e9b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:35.358590 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 339/681] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:21:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:35.358590 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec510f-0edb-4e00-a811-ce77f0bc9e9b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:35.404255 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:35.408243 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e7d14-e124-4017-ae53-72886249963f None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:35.448964 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b83b4b-606f-4d31-88ad-d605366150b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:35.449924 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b83b4b-606f-4d31-88ad-d605366150b1 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:35.450452 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b83b4b-606f-4d31-88ad-d605366150b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:35.450956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b83b4b-606f-4d31-88ad-d605366150b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:35.451882 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b83b4b-606f-4d31-88ad-d605366150b1 None admin] RBAC: Adding query filter params (role.id=b6727ed962fa41109b0a2d6851551f49, scope.project.id=36ce80c4e0d64562a9ab3961be76fad9, user.id=b78747bb9140457896fe62d4f537592c) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:35.452415 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b83b4b-606f-4d31-88ad-d605366150b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:35.454268 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b83b4b-606f-4d31-88ad-d605366150b1 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:35.466649 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 343/682] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:21:35 2022] GET /identity/v3/role_assignments?user.id=b78747bb9140457896fe62d4f537592c&scope.project.id=36ce80c4e0d64562a9ab3961be76fad9&role.id=b6727ed962fa41109b0a2d6851551f49 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:35.468017 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b83b4b-606f-4d31-88ad-d605366150b1 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:35.482864 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec510f-0edb-4e00-a811-ce77f0bc9e9b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:35.525226 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add76124-3285-4fcc-9040-fd86d478425c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:35.525869 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add76124-3285-4fcc-9040-fd86d478425c None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:35.526202 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add76124-3285-4fcc-9040-fd86d478425c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:35.526528 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add76124-3285-4fcc-9040-fd86d478425c None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:35.527316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add76124-3285-4fcc-9040-fd86d478425c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:35.527632 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add76124-3285-4fcc-9040-fd86d478425c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:35.529316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add76124-3285-4fcc-9040-fd86d478425c None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:35.534632 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:35.552402 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 340/683] 10.222.0.18 () {56 vars in 1135 bytes} [Thu Sep 1 16:21:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:35.558859 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add76124-3285-4fcc-9040-fd86d478425c None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:35.679313 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b83b4b-606f-4d31-88ad-d605366150b1 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:35.712022 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add76124-3285-4fcc-9040-fd86d478425c None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:35.727205 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa66055-0465-44d5-82e3-53b0e3d290d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:35.727817 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa66055-0465-44d5-82e3-53b0e3d290d1 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:35.728097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa66055-0465-44d5-82e3-53b0e3d290d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:35.728369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa66055-0465-44d5-82e3-53b0e3d290d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:35.729736 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa66055-0465-44d5-82e3-53b0e3d290d1 None admin] RBAC: Adding query filter params (role.id=b6727ed962fa41109b0a2d6851551f49, scope.project.id=36ce80c4e0d64562a9ab3961be76fad9, user.id=ffa0f2160a934906b59e6f845edb0776) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:35.730044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa66055-0465-44d5-82e3-53b0e3d290d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:35.731733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa66055-0465-44d5-82e3-53b0e3d290d1 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:35.738651 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 344/684] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:21:35 2022] GET /identity/v3/role_assignments?user.id=ffa0f2160a934906b59e6f845edb0776&scope.project.id=36ce80c4e0d64562a9ab3961be76fad9&role.id=b6727ed962fa41109b0a2d6851551f49 => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:35.753318 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae484c-5a60-4d68-96e0-7269e85d9d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:35.753945 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae484c-5a60-4d68-96e0-7269e85d9d17 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:35.754230 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae484c-5a60-4d68-96e0-7269e85d9d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:35.754554 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae484c-5a60-4d68-96e0-7269e85d9d17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:35.823263 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:35.834973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae484c-5a60-4d68-96e0-7269e85d9d17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:35.836463 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae484c-5a60-4d68-96e0-7269e85d9d17 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:35.950583 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 341/685] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:21:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:21:35.956217 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa66055-0465-44d5-82e3-53b0e3d290d1 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:35.996288 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:36.074662 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa66055-0465-44d5-82e3-53b0e3d290d1 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:36.112789 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177596dd-91a3-4128-be47-b3a780b31506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:36.113746 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177596dd-91a3-4128-be47-b3a780b31506 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:36.114028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177596dd-91a3-4128-be47-b3a780b31506 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:36.114319 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177596dd-91a3-4128-be47-b3a780b31506 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:36.126079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177596dd-91a3-4128-be47-b3a780b31506 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:36.127698 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177596dd-91a3-4128-be47-b3a780b31506 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:36.142705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-177596dd-91a3-4128-be47-b3a780b31506 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:36.142705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:36.142705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:36.142705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:36.142705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:36.142705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:36.142705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:36.142705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:36.142705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:36.142705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:36.142705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:36.142705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:36.142705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:36.142705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:36.142705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:36.142705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:36.143641 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:36.143641 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:36.143641 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:36.143641 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:36.143641 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:36.143641 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:36.143641 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:36.143641 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:36.143641 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:36.143641 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:36.143641 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:36.143641 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:36.143641 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:36.143641 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:36.143641 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:36.143641 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:36.143641 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:36.144524 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:36.144524 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:36.144524 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:36.148360 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 345/686] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:36.149774 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae484c-5a60-4d68-96e0-7269e85d9d17 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:36.280878 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:36.290544 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae484c-5a60-4d68-96e0-7269e85d9d17 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:36.326830 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6617f-3224-424d-826c-4f03c312ba6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:36.327851 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6617f-3224-424d-826c-4f03c312ba6f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:36.328328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6617f-3224-424d-826c-4f03c312ba6f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:36.328795 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6617f-3224-424d-826c-4f03c312ba6f None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:36.329631 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f6617f-3224-424d-826c-4f03c312ba6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:36.330141 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f6617f-3224-424d-826c-4f03c312ba6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:36.331709 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f6617f-3224-424d-826c-4f03c312ba6f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:36.344442 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 342/687] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:36.349794 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177596dd-91a3-4128-be47-b3a780b31506 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:36.447528 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cc4ad-d2b6-4e4b-a09c-f77281d3f761 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:36.448723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cc4ad-d2b6-4e4b-a09c-f77281d3f761 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:36.449197 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cc4ad-d2b6-4e4b-a09c-f77281d3f761 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:36.450994 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 343/688] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:36.456334 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191c658-363d-4248-b1bd-af921b65d6f7 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:36.456877 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191c658-363d-4248-b1bd-af921b65d6f7 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:36.457371 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191c658-363d-4248-b1bd-af921b65d6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:36.473906 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:36.489946 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-7191c658-363d-4248-b1bd-af921b65d6f7 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:36.491725 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7191c658-363d-4248-b1bd-af921b65d6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:36.494068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177596dd-91a3-4128-be47-b3a780b31506 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:36.529726 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61329aca-e4bd-4853-a194-d6d2b9553a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:36.530397 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61329aca-e4bd-4853-a194-d6d2b9553a82 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:36.530673 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61329aca-e4bd-4853-a194-d6d2b9553a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:36.530993 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61329aca-e4bd-4853-a194-d6d2b9553a82 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:36.546681 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61329aca-e4bd-4853-a194-d6d2b9553a82 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:36.548324 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61329aca-e4bd-4853-a194-d6d2b9553a82 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:36.568039 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-61329aca-e4bd-4853-a194-d6d2b9553a82 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:36.568039 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:36.568039 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:36.568039 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:36.568039 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:36.568039 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:36.568039 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:36.568039 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:36.568039 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:36.568039 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:36.568039 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:36.568039 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:36.568039 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:21:36.568039 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:21:36.568039 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:21:36.568935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:21:36.568935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:36.568935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:36.568935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:36.568935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:36.568935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:36.568935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:36.568935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:36.568935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:36.568935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:36.568935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:36.568935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:36.568935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:36.568935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:36.568935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:36.568935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:36.568935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:36.569794 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:21:36.569794 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:21:36.569794 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:21:36.569794 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:21:36.569794 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:21:36.569794 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:21:36.569794 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:36.569794 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:36.569794 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 346/689] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:36.576016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61329aca-e4bd-4853-a194-d6d2b9553a82 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:36.592640 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 344/690] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:21:36.706469 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61329aca-e4bd-4853-a194-d6d2b9553a82 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:36.740991 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4617c0c0-a7df-4d6a-907f-2833ec755414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:36.742029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617c0c0-a7df-4d6a-907f-2833ec755414 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:36.742548 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617c0c0-a7df-4d6a-907f-2833ec755414 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:36.743035 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617c0c0-a7df-4d6a-907f-2833ec755414 None admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:36.743915 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4617c0c0-a7df-4d6a-907f-2833ec755414 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:36.744432 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4617c0c0-a7df-4d6a-907f-2833ec755414 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:36.746842 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4617c0c0-a7df-4d6a-907f-2833ec755414 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:36.756604 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:36.761237 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 347/691] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:21:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:21:36.767738 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191c658-363d-4248-b1bd-af921b65d6f7 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:36.853344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:36.880148 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191c658-363d-4248-b1bd-af921b65d6f7 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:36.927096 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5136f0-8188-484d-967b-d022edd183da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:36.927707 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5136f0-8188-484d-967b-d022edd183da None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:36.927984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5136f0-8188-484d-967b-d022edd183da None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:36.928253 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5136f0-8188-484d-967b-d022edd183da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:36.928979 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5136f0-8188-484d-967b-d022edd183da None admin] RBAC: Adding query filter params (role.id=b6727ed962fa41109b0a2d6851551f49, scope.project.id=36ce80c4e0d64562a9ab3961be76fad9, user.id=8137f6b2ced040bd9752239c7e987f03) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:36.929287 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5136f0-8188-484d-967b-d022edd183da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:36.930726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5136f0-8188-484d-967b-d022edd183da None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:36.941336 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 345/692] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:21:36 2022] GET /identity/v3/role_assignments?user.id=8137f6b2ced040bd9752239c7e987f03&scope.project.id=36ce80c4e0d64562a9ab3961be76fad9&role.id=b6727ed962fa41109b0a2d6851551f49 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:37.009327 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4617c0c0-a7df-4d6a-907f-2833ec755414 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:37.126619 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4617c0c0-a7df-4d6a-907f-2833ec755414 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:37.162345 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec18e0-8b2c-4167-b9fd-c413f1da4bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:37.162987 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec18e0-8b2c-4167-b9fd-c413f1da4bcb None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:37.163276 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec18e0-8b2c-4167-b9fd-c413f1da4bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:37.163547 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec18e0-8b2c-4167-b9fd-c413f1da4bcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:37.174561 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ec18e0-8b2c-4167-b9fd-c413f1da4bcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:37.176113 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ec18e0-8b2c-4167-b9fd-c413f1da4bcb None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:37.191538 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-48ec18e0-8b2c-4167-b9fd-c413f1da4bcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:37.191538 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:37.191538 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:37.191538 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:37.191538 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:37.191538 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:37.191538 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:37.191538 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:37.191538 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:37.191538 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:37.191538 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:37.191538 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:37.191538 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:37.191538 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:37.191538 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:37.191538 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:37.192557 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:37.192557 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:37.192557 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:37.192557 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:37.192557 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:37.192557 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:37.192557 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:37.192557 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:37.192557 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:37.192557 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:37.192557 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:37.192557 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:37.192557 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:37.192557 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:37.192557 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:37.192557 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:37.192557 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:37.193507 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:37.193507 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:37.193507 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:37.193507 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 348/693] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:37.199463 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5136f0-8188-484d-967b-d022edd183da None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:37.316413 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5136f0-8188-484d-967b-d022edd183da None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:37.356766 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71801d51-d65e-43c1-a398-dfd905a84ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:37.357381 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71801d51-d65e-43c1-a398-dfd905a84ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:37.357662 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71801d51-d65e-43c1-a398-dfd905a84ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:37.357935 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71801d51-d65e-43c1-a398-dfd905a84ae6 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:37.358611 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71801d51-d65e-43c1-a398-dfd905a84ae6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:37.358913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71801d51-d65e-43c1-a398-dfd905a84ae6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:37.360264 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71801d51-d65e-43c1-a398-dfd905a84ae6 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:37.378493 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 346/694] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:37.383922 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec18e0-8b2c-4167-b9fd-c413f1da4bcb None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:37.464137 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:37.490425 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868edd36-48e6-4a19-b78a-9ae07bbc3ae6 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:37.490743 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868edd36-48e6-4a19-b78a-9ae07bbc3ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:37.490983 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868edd36-48e6-4a19-b78a-9ae07bbc3ae6 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:37.492676 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 347/695] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:21:37.498419 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29770c12-82b3-4806-9c7d-655be58a7996 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:37.498704 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29770c12-82b3-4806-9c7d-655be58a7996 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:37.498981 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29770c12-82b3-4806-9c7d-655be58a7996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:37.524014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec18e0-8b2c-4167-b9fd-c413f1da4bcb None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:37.530594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:37.542968 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-29770c12-82b3-4806-9c7d-655be58a7996 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:37.544249 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-29770c12-82b3-4806-9c7d-655be58a7996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:37.583008 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d8b76e-14a3-44c0-9f77-deea510e63ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:37.583644 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8b76e-14a3-44c0-9f77-deea510e63ac None admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:37.583927 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8b76e-14a3-44c0-9f77-deea510e63ac None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:37.584208 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8b76e-14a3-44c0-9f77-deea510e63ac None admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:37.584864 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d8b76e-14a3-44c0-9f77-deea510e63ac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:37.586432 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d8b76e-14a3-44c0-9f77-deea510e63ac None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:37.631321 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 349/696] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 1 16:21:37 2022] POST /identity/v3/users => generated 254 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:21:37.672137 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 348/697] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:21:37.981946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd17119-abdd-4dfd-8c56-208b44104319 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:37.982661 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd17119-abdd-4dfd-8c56-208b44104319 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:37.983154 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd17119-abdd-4dfd-8c56-208b44104319 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:37.984909 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 350/698] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:37.989944 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94d320-6694-42d8-8050-45f26255bd53 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:37.990397 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94d320-6694-42d8-8050-45f26255bd53 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:37.990574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94d320-6694-42d8-8050-45f26255bd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:38.000214 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd17119-abdd-4dfd-8c56-208b44104319 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:38.013911 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-4c94d320-6694-42d8-8050-45f26255bd53 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:38.015138 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-4c94d320-6694-42d8-8050-45f26255bd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:38.095312 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 349/699] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:21:38.149330 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd17119-abdd-4dfd-8c56-208b44104319 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:38.186634 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:38.195751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e6206-9380-474b-92c7-b86821bff6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:38.196489 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e6206-9380-474b-92c7-b86821bff6fb None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:38.196775 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e6206-9380-474b-92c7-b86821bff6fb None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:38.197052 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e6206-9380-474b-92c7-b86821bff6fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:38.211359 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e6206-9380-474b-92c7-b86821bff6fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:38.217044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-b43e6206-9380-474b-92c7-b86821bff6fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:21:38.217044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:38.217044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:38.217044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:38.217044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:38.217044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:38.217044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:38.217044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:38.217044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:38.217044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:38.217044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:38.217044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:38.217044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:38.217044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:38.217044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:38.217044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:38.217950 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:38.217950 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:38.217950 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:38.217950 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:38.217950 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:38.217950 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:38.217950 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:38.217950 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:38.217950 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:38.217950 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:38.217950 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:38.217950 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:38.217950 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:38.217950 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:38.217950 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:38.217950 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:38.219058 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:38.219058 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:38.219058 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:38.219058 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:38.219058 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:38.219058 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:38.219058 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:38.219058 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:38.219058 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:38.219058 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:38.219058 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:21:38.219058 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:38.219058 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 351/700] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:21:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:38.223937 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c94d320-6694-42d8-8050-45f26255bd53 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:38.235944 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d6b0d-cf34-4cb9-9449-87d4ab1b85f2 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:38.237368 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d6b0d-cf34-4cb9-9449-87d4ab1b85f2 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:38.237657 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d6b0d-cf34-4cb9-9449-87d4ab1b85f2 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:38.239285 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 352/701] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:38 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:38.247488 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6895e50-87de-4ee7-8465-58cba2d38e73 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:38.247778 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6895e50-87de-4ee7-8465-58cba2d38e73 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:38.248059 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6895e50-87de-4ee7-8465-58cba2d38e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:38.279257 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-e6895e50-87de-4ee7-8465-58cba2d38e73 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:38.280477 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-e6895e50-87de-4ee7-8465-58cba2d38e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:38.318579 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:38.364685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c94d320-6694-42d8-8050-45f26255bd53 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:38.404690 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35527d73-2cff-4785-856e-b3560086b31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:38.405304 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35527d73-2cff-4785-856e-b3560086b31b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:38.405598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35527d73-2cff-4785-856e-b3560086b31b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:38.405864 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35527d73-2cff-4785-856e-b3560086b31b None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:38.406560 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35527d73-2cff-4785-856e-b3560086b31b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:38.406868 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35527d73-2cff-4785-856e-b3560086b31b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:38.408296 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35527d73-2cff-4785-856e-b3560086b31b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:38.409115 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 353/702] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:21:38.411101 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6895e50-87de-4ee7-8465-58cba2d38e73 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:38.418856 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 350/703] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:21:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:38.429622 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35527d73-2cff-4785-856e-b3560086b31b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:38.527253 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:38.531625 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6895e50-87de-4ee7-8465-58cba2d38e73 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:38.571646 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9444aadf-cd41-4d1b-9c25-7db174ec860a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:38.572536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444aadf-cd41-4d1b-9c25-7db174ec860a None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:38.573818 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444aadf-cd41-4d1b-9c25-7db174ec860a None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:38.574340 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444aadf-cd41-4d1b-9c25-7db174ec860a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:38.575241 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9444aadf-cd41-4d1b-9c25-7db174ec860a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:38.577028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9444aadf-cd41-4d1b-9c25-7db174ec860a None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:38.590043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-9444aadf-cd41-4d1b-9c25-7db174ec860a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:21:38.590043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:38.590043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:21:38.590043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:21:38.590043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:21:38.590043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:21:38.590043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:21:38.590043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:21:38.590043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:21:38.590043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:38.590043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:21:38.590043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:38.590043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:38.590043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:38.590043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:38.590043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:38.590043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:38.591322 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:38.591322 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:38.591322 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:38.591322 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:38.591322 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:38.591322 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:38.591322 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:21:38.591322 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:21:38.591322 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:21:38.591322 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:21:38.591322 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:38.591322 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:38.591322 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:38.591322 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:38.591322 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:38.591322 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:38.592390 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:38.592390 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:38.592390 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:38.592390 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:38.592390 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:38.592390 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:38.592390 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:38.592390 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:38.592390 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:38.592390 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:38.592390 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:21:38.592390 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:21:38.592390 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:21:38.592390 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:38.595655 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 354/704] 10.222.0.18 () {56 vars in 1129 bytes} [Thu Sep 1 16:21:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:38.601662 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9444aadf-cd41-4d1b-9c25-7db174ec860a None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:38.657158 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35527d73-2cff-4785-856e-b3560086b31b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:38.707435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:38.716538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2610bb-f1b2-415e-80d5-36d5ab03cac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:38.717554 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2610bb-f1b2-415e-80d5-36d5ab03cac0 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:38.718321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2610bb-f1b2-415e-80d5-36d5ab03cac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:38.718947 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2610bb-f1b2-415e-80d5-36d5ab03cac0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:38.721362 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9444aadf-cd41-4d1b-9c25-7db174ec860a None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:38.732629 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2610bb-f1b2-415e-80d5-36d5ab03cac0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:38.734974 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2610bb-f1b2-415e-80d5-36d5ab03cac0 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:38.755705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-3d2610bb-f1b2-415e-80d5-36d5ab03cac0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:38.755705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:38.755705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:38.755705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:38.755705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:38.755705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:38.755705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:38.755705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:38.755705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:38.755705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:38.755705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:38.755705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:38.755705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:38.755705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:38.755705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:38.755705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:38.756846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:38.756846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:38.756846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:38.756846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:38.756846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:38.756846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:38.756846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:38.756846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:38.756846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:38.756846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:38.756846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:38.756846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:38.756846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:38.756846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:38.756846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:38.756846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:38.756846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:38.757933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:38.757933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:38.757933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:38.759830 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 351/705] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:38.760767 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2610bb-f1b2-415e-80d5-36d5ab03cac0 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:38.767466 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7a9ec-28b8-40ff-ad9a-277e83615090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:38.768385 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7a9ec-28b8-40ff-ad9a-277e83615090 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:38.769094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7a9ec-28b8-40ff-ad9a-277e83615090 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:38.769564 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7a9ec-28b8-40ff-ad9a-277e83615090 None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:38.770366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e7a9ec-28b8-40ff-ad9a-277e83615090 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:38.770835 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e7a9ec-28b8-40ff-ad9a-277e83615090 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:38.772457 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e7a9ec-28b8-40ff-ad9a-277e83615090 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:38.777844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 355/706] 10.222.0.18 () {56 vars in 1124 bytes} [Thu Sep 1 16:21:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:21:38.783352 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7a9ec-28b8-40ff-ad9a-277e83615090 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:38.814579 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:38.869215 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2610bb-f1b2-415e-80d5-36d5ab03cac0 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:38.892352 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7a9ec-28b8-40ff-ad9a-277e83615090 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:38.905140 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd6ea5-f146-4d8b-a9c6-3437e04da0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:38.905895 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd6ea5-f146-4d8b-a9c6-3437e04da0ae None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:38.906187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd6ea5-f146-4d8b-a9c6-3437e04da0ae None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:38.906694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd6ea5-f146-4d8b-a9c6-3437e04da0ae None admin] PATH_INFO: `/v3/services/network` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:38.907078 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd6ea5-f146-4d8b-a9c6-3437e04da0ae None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:38.908442 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd6ea5-f146-4d8b-a9c6-3437e04da0ae None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:38.920483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-6bcd6ea5-f146-4d8b-a9c6-3437e04da0ae None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:21:38.920483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:38.920483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:21:38.920483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:21:38.920483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:21:38.920483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:21:38.920483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:21:38.920483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:21:38.920483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:21:38.920483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:38.920483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:21:38.920483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:38.920483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:38.920483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:38.920483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:38.920483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:38.920483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:38.921699 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:38.921699 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:38.921699 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:38.921699 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:38.921699 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:38.921699 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:38.921699 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:21:38.921699 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:21:38.921699 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:21:38.921699 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:21:38.921699 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:38.921699 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:38.921699 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:38.921699 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:38.921699 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:38.921699 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:38.922598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:38.922598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:38.922598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:38.922598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:38.922598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:38.922598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:38.922598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:38.922598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:38.922598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:38.922598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:38.922598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:21:38.922598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:21:38.922598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:21:38.922598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:38.926450 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 352/707] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:38 2022] GET /identity/v3/services/network => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:38.927302 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd6ea5-f146-4d8b-a9c6-3437e04da0ae None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:38.938332 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9d00a-6e7d-4aa2-99ad-9cdf1c4890e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:38.939195 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9d00a-6e7d-4aa2-99ad-9cdf1c4890e7 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:38.940064 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9d00a-6e7d-4aa2-99ad-9cdf1c4890e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:38.940495 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9d00a-6e7d-4aa2-99ad-9cdf1c4890e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:38.941704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b9d00a-6e7d-4aa2-99ad-9cdf1c4890e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:38.942452 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b9d00a-6e7d-4aa2-99ad-9cdf1c4890e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:38.945194 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b9d00a-6e7d-4aa2-99ad-9cdf1c4890e7 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:38.962492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 356/708] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:38.963378 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9d00a-6e7d-4aa2-99ad-9cdf1c4890e7 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:38.988469 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:39.056519 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:39.060923 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd6ea5-f146-4d8b-a9c6-3437e04da0ae None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:39.075915 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9d00a-6e7d-4aa2-99ad-9cdf1c4890e7 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:39.105828 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12184b42-2ee8-400a-8d79-6741c723a3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:39.106637 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12184b42-2ee8-400a-8d79-6741c723a3cf None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:39.107159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12184b42-2ee8-400a-8d79-6741c723a3cf None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:39.107554 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12184b42-2ee8-400a-8d79-6741c723a3cf None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:39.108593 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12184b42-2ee8-400a-8d79-6741c723a3cf None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:39.108995 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12184b42-2ee8-400a-8d79-6741c723a3cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:39.110763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12184b42-2ee8-400a-8d79-6741c723a3cf None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:39.113395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442876b-0b82-4445-b3a9-e3823c8acfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:39.114177 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442876b-0b82-4445-b3a9-e3823c8acfe3 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:39.114648 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442876b-0b82-4445-b3a9-e3823c8acfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:39.115075 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442876b-0b82-4445-b3a9-e3823c8acfe3 None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:39.115867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442876b-0b82-4445-b3a9-e3823c8acfe3 None admin] RBAC: Adding query filter params (name=network) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:39.116326 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442876b-0b82-4445-b3a9-e3823c8acfe3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:39.117844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442876b-0b82-4445-b3a9-e3823c8acfe3 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:39.122438 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 353/709] 10.222.0.18 () {56 vars in 1124 bytes} [Thu Sep 1 16:21:38 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:39.124555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12184b42-2ee8-400a-8d79-6741c723a3cf None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:39.130296 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 357/710] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Sep 1 16:21:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:39.147824 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442876b-0b82-4445-b3a9-e3823c8acfe3 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:39.320880 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442876b-0b82-4445-b3a9-e3823c8acfe3 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:39.323139 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12184b42-2ee8-400a-8d79-6741c723a3cf None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:39.333171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:39.358211 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa4e50-8094-4115-be6e-d9e6478dc4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:39.359140 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa4e50-8094-4115-be6e-d9e6478dc4a4 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:39.359270 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa4e50-8094-4115-be6e-d9e6478dc4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:39.359478 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa4e50-8094-4115-be6e-d9e6478dc4a4 None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:39.360086 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aa4e50-8094-4115-be6e-d9e6478dc4a4 None admin] RBAC: Adding query filter params (type=network) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:39.360390 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aa4e50-8094-4115-be6e-d9e6478dc4a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:39.361758 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aa4e50-8094-4115-be6e-d9e6478dc4a4 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:39.371060 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 358/711] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Sep 1 16:21:39 2022] GET /identity/v3/services?type=network => generated 124 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:21:39.377035 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53d474-f0b4-4347-8813-0137bb2bfeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:39.378064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53d474-f0b4-4347-8813-0137bb2bfeca None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:39.378620 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53d474-f0b4-4347-8813-0137bb2bfeca None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:39.379178 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53d474-f0b4-4347-8813-0137bb2bfeca None admin] PATH_INFO: `/v3/users/nova` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:39.390413 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d53d474-f0b4-4347-8813-0137bb2bfeca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:39.392312 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d53d474-f0b4-4347-8813-0137bb2bfeca None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:39.408166 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-4d53d474-f0b4-4347-8813-0137bb2bfeca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:21:39.408166 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:39.408166 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:39.408166 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:39.408166 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:39.408166 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:39.408166 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:39.408166 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:39.408166 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:39.408166 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:39.408166 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:39.408166 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:39.408166 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:21:39.408166 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:21:39.408166 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:21:39.408166 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:21:39.409173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:39.409173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:39.409173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:21:39.409173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:39.409173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:21:39.409173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:39.409173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:39.409173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:39.409173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:39.409173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:39.409173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:39.409173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:39.409173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:39.409173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:39.409173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:39.409173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:39.409173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:39.409173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:39.410105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:39.410105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:39.410105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:21:39.410105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:21:39.410105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:21:39.410105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:21:39.410105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:21:39.410105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:21:39.410105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:21:39.410105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:39.412986 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 354/712] 10.222.0.18 () {56 vars in 1093 bytes} [Thu Sep 1 16:21:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:39.418954 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa4e50-8094-4115-be6e-d9e6478dc4a4 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:39.438929 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:39.533705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764eb7c-4f65-4d30-9b02-a27ab097dc15 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:39.533993 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764eb7c-4f65-4d30-9b02-a27ab097dc15 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:39.534294 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764eb7c-4f65-4d30-9b02-a27ab097dc15 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:39.535673 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 355/713] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:21:39.541600 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087092d-23fb-4cc9-a05f-164591c6ca5f None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:39.542718 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087092d-23fb-4cc9-a05f-164591c6ca5f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:39.543216 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087092d-23fb-4cc9-a05f-164591c6ca5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:39.552205 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa4e50-8094-4115-be6e-d9e6478dc4a4 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:39.575134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-b087092d-23fb-4cc9-a05f-164591c6ca5f None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:39.576552 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b087092d-23fb-4cc9-a05f-164591c6ca5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:39.596152 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdf237-732f-4bfe-a59b-78f7de5f44b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:39.597161 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdf237-732f-4bfe-a59b-78f7de5f44b2 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:39.597701 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdf237-732f-4bfe-a59b-78f7de5f44b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:39.598174 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdf237-732f-4bfe-a59b-78f7de5f44b2 None admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:39.599157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbdf237-732f-4bfe-a59b-78f7de5f44b2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:39.599676 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbdf237-732f-4bfe-a59b-78f7de5f44b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:39.601321 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbdf237-732f-4bfe-a59b-78f7de5f44b2 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:39.623015 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 359/714] 10.222.0.18 () {56 vars in 1133 bytes} [Thu Sep 1 16:21:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:39.629574 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdf237-732f-4bfe-a59b-78f7de5f44b2 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:39.665486 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 356/715] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:21:39.711576 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:39.743008 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdf237-732f-4bfe-a59b-78f7de5f44b2 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:39.794034 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01645e-11dd-4a4e-9d93-69828bd712df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:39.794683 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01645e-11dd-4a4e-9d93-69828bd712df None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:39.794969 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01645e-11dd-4a4e-9d93-69828bd712df None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:39.795242 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01645e-11dd-4a4e-9d93-69828bd712df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:39.892739 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01645e-11dd-4a4e-9d93-69828bd712df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:39.894113 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01645e-11dd-4a4e-9d93-69828bd712df None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:39.942638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b087092d-23fb-4cc9-a05f-164591c6ca5f None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:39.974646 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:40.038848 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 360/716] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:21:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 16:21:40.045628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01645e-11dd-4a4e-9d93-69828bd712df None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:40.059938 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:40.063568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b087092d-23fb-4cc9-a05f-164591c6ca5f None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:40.104056 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f0893-bd90-4f09-824d-e49f08e8e121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:40.105110 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f0893-bd90-4f09-824d-e49f08e8e121 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:40.105677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f0893-bd90-4f09-824d-e49f08e8e121 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:40.106169 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f0893-bd90-4f09-824d-e49f08e8e121 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:40.118359 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715f0893-bd90-4f09-824d-e49f08e8e121 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:40.125319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-715f0893-bd90-4f09-824d-e49f08e8e121 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:21:40.125319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:40.125319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:40.125319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:40.125319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:40.125319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:40.125319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:40.125319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:40.125319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:40.125319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:40.125319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:40.125319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:40.125319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:40.125319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:40.125319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:40.125319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:40.126355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:40.126355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:40.126355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:40.126355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:40.126355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:40.126355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:40.126355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:40.126355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:40.126355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:40.126355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:40.126355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:40.126355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:40.126355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:40.126355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:40.126355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:40.126355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:40.127178 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:40.127178 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:40.127178 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:40.127178 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:40.127178 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:40.127178 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:40.127178 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:40.127178 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:40.127178 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:40.127178 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:40.127178 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:21:40.127178 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:40.127809 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 357/717] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:21:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:40.135958 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f0893-bd90-4f09-824d-e49f08e8e121 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:40.211645 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01645e-11dd-4a4e-9d93-69828bd712df None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:40.249127 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da00598-d528-455e-83f0-8f03234a179f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:40.250220 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da00598-d528-455e-83f0-8f03234a179f None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:40.250588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da00598-d528-455e-83f0-8f03234a179f None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:40.250937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da00598-d528-455e-83f0-8f03234a179f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:40.262588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da00598-d528-455e-83f0-8f03234a179f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:40.264143 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da00598-d528-455e-83f0-8f03234a179f None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:40.265276 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f0893-bd90-4f09-824d-e49f08e8e121 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:40.280227 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-9da00598-d528-455e-83f0-8f03234a179f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:40.280227 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:40.280227 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:40.280227 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:40.280227 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:40.280227 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:40.280227 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:40.280227 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:40.280227 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:40.280227 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:40.280227 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:40.280227 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:40.280227 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:40.280227 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:40.280227 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:40.280227 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:40.281181 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:40.281181 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:40.281181 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:40.281181 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:40.281181 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:40.281181 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:40.281181 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:40.281181 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:40.281181 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:40.281181 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:40.281181 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:40.281181 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:40.281181 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:40.281181 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:40.281181 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:40.281181 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:40.281181 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:40.282430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:40.282430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:40.282430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:40.282430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 361/718] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:21:40.283387 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b87c1-970b-4938-b6e3-4a928969207d None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:40.283741 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b87c1-970b-4938-b6e3-4a928969207d None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:40.284184 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b87c1-970b-4938-b6e3-4a928969207d None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:40.289705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 362/719] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:40.290574 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b87c1-970b-4938-b6e3-4a928969207d None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:40.310599 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e99fe0-47ef-463d-acee-cff93c262310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:40.311694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e99fe0-47ef-463d-acee-cff93c262310 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:40.312542 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e99fe0-47ef-463d-acee-cff93c262310 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:40.313004 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e99fe0-47ef-463d-acee-cff93c262310 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:40.313839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e99fe0-47ef-463d-acee-cff93c262310 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:40.314352 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e99fe0-47ef-463d-acee-cff93c262310 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:40.316373 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e99fe0-47ef-463d-acee-cff93c262310 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:40.319223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:40.331519 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 358/720] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:21:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:21:40.333142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ebff1f-7153-4dda-9bb9-d43fdd63a1e2 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:40.333644 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ebff1f-7153-4dda-9bb9-d43fdd63a1e2 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:40.334090 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ebff1f-7153-4dda-9bb9-d43fdd63a1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:40.358364 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-d8ebff1f-7153-4dda-9bb9-d43fdd63a1e2 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:40.360492 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d8ebff1f-7153-4dda-9bb9-d43fdd63a1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:40.382336 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:40.415836 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b87c1-970b-4938-b6e3-4a928969207d None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:40.451965 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e2a27-aafe-4cbf-b52b-1b356b187708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:40.452687 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e2a27-aafe-4cbf-b52b-1b356b187708 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:40.453031 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e2a27-aafe-4cbf-b52b-1b356b187708 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:40.453383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e2a27-aafe-4cbf-b52b-1b356b187708 None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:40.454090 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08e2a27-aafe-4cbf-b52b-1b356b187708 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:40.454487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08e2a27-aafe-4cbf-b52b-1b356b187708 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:40.455943 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08e2a27-aafe-4cbf-b52b-1b356b187708 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:40.460684 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 359/721] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:21:40.461875 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ebff1f-7153-4dda-9bb9-d43fdd63a1e2 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:40.480765 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 363/722] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:40.485710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e2a27-aafe-4cbf-b52b-1b356b187708 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:40.581062 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ebff1f-7153-4dda-9bb9-d43fdd63a1e2 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:40.621726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dafaed-a821-43ee-b37d-947588b32721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:40.623263 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dafaed-a821-43ee-b37d-947588b32721 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:40.623873 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dafaed-a821-43ee-b37d-947588b32721 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:40.624325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dafaed-a821-43ee-b37d-947588b32721 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:40.636621 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dafaed-a821-43ee-b37d-947588b32721 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:40.638935 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dafaed-a821-43ee-b37d-947588b32721 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:40.655615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-31dafaed-a821-43ee-b37d-947588b32721 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:40.655615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:40.655615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:40.655615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:40.655615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:40.655615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:40.655615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:40.655615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:40.655615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:40.655615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:40.655615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:40.655615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:40.655615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:40.655615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:40.655615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:40.655615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:40.656596 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:40.656596 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:40.656596 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:40.656596 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:40.656596 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:40.656596 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:40.656596 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:40.656596 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:40.656596 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:40.656596 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:40.656596 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:40.656596 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:40.656596 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:40.656596 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:40.656596 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:40.656596 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:40.656596 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:40.657463 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:40.657463 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:40.657463 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:40.659845 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 360/723] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:40.670561 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dafaed-a821-43ee-b37d-947588b32721 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:40.674896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:40.685314 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:40.728727 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e2a27-aafe-4cbf-b52b-1b356b187708 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:40.786916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7696ca8a-930a-466d-9a6c-9da6194ab045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:40.787532 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696ca8a-930a-466d-9a6c-9da6194ab045 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:40.787811 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696ca8a-930a-466d-9a6c-9da6194ab045 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:40.788092 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696ca8a-930a-466d-9a6c-9da6194ab045 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:40.794968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7696ca8a-930a-466d-9a6c-9da6194ab045 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:40.796415 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7696ca8a-930a-466d-9a6c-9da6194ab045 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:40.800917 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dafaed-a821-43ee-b37d-947588b32721 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:40.808703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-7696ca8a-930a-466d-9a6c-9da6194ab045 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:40.808703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:40.808703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:40.808703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:40.808703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:40.808703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:40.808703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:40.808703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:40.808703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:40.808703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:40.808703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:40.808703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:40.808703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:21:40.808703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:21:40.808703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:21:40.809682 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:21:40.809682 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:40.809682 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:40.809682 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:40.809682 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:40.809682 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:40.809682 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:40.809682 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:40.809682 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:40.809682 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:40.809682 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:40.809682 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:40.809682 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:40.809682 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:40.809682 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:40.809682 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:40.809682 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:40.811154 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:21:40.811154 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:21:40.811154 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:21:40.811154 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:21:40.811154 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:21:40.811154 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:21:40.811154 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:40.811154 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:40.811154 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 364/724] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:21:40.811154 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7696ca8a-930a-466d-9a6c-9da6194ab045 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:40.863335 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f7e731-db7d-4885-90d4-3daa868f93fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:40.864348 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7e731-db7d-4885-90d4-3daa868f93fb None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:40.864639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7e731-db7d-4885-90d4-3daa868f93fb None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:40.864920 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7e731-db7d-4885-90d4-3daa868f93fb None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:40.865576 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f7e731-db7d-4885-90d4-3daa868f93fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:40.865885 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f7e731-db7d-4885-90d4-3daa868f93fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:40.867821 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f7e731-db7d-4885-90d4-3daa868f93fb None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:40.888118 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 361/725] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:21:40.889087 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f7e731-db7d-4885-90d4-3daa868f93fb None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:40.924020 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7696ca8a-930a-466d-9a6c-9da6194ab045 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:40.960124 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:40.968743 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c42298-5b31-4c47-a357-005d348637d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:40.969684 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c42298-5b31-4c47-a357-005d348637d7 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:40.970119 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c42298-5b31-4c47-a357-005d348637d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:40.970577 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c42298-5b31-4c47-a357-005d348637d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:40.982474 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c42298-5b31-4c47-a357-005d348637d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:40.988426 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-34c42298-5b31-4c47-a357-005d348637d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:40.988426 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:40.988426 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:40.988426 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:40.988426 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:40.988426 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:40.988426 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:40.988426 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:40.988426 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:40.988426 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:40.988426 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:40.988426 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:40.988426 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:40.988426 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:40.988426 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:40.988426 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:40.989515 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:40.989515 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:40.989515 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:40.989515 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:40.989515 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:40.989515 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:40.989515 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:40.989515 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:40.989515 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:40.989515 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:40.989515 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:40.989515 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:40.989515 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:40.989515 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:40.989515 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:40.989515 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:40.990588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:40.990588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:40.990588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:40.990588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:40.990588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:40.990588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:40.990588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:40.990588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:40.990588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:40.990588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:40.990588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:40.990588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:40.992723 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 365/726] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 1 16:21:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:40.993621 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c42298-5b31-4c47-a357-005d348637d7 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:40.998123 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:41.005909 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f7e731-db7d-4885-90d4-3daa868f93fb None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:41.041269 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd61e469-73cb-4198-9f09-f0f4791bd07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:41.042479 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61e469-73cb-4198-9f09-f0f4791bd07e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:41.042757 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61e469-73cb-4198-9f09-f0f4791bd07e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:41.043049 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61e469-73cb-4198-9f09-f0f4791bd07e None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:41.043779 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd61e469-73cb-4198-9f09-f0f4791bd07e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:41.044084 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd61e469-73cb-4198-9f09-f0f4791bd07e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:41.045708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd61e469-73cb-4198-9f09-f0f4791bd07e None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:41.062288 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 362/727] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:21:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:41.062288 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd61e469-73cb-4198-9f09-f0f4791bd07e None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:41.109289 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c42298-5b31-4c47-a357-005d348637d7 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:41.145454 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f2607-4056-482f-9a6e-85d26e56d850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:41.146189 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f2607-4056-482f-9a6e-85d26e56d850 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:41.146816 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f2607-4056-482f-9a6e-85d26e56d850 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:41.146816 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f2607-4056-482f-9a6e-85d26e56d850 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:41.154948 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f2607-4056-482f-9a6e-85d26e56d850 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:41.156494 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f2607-4056-482f-9a6e-85d26e56d850 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:41.166183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd61e469-73cb-4198-9f09-f0f4791bd07e None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:41.168840 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-899f2607-4056-482f-9a6e-85d26e56d850 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:21:41.168840 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:41.168840 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:41.168840 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:41.168840 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:41.168840 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:41.168840 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:41.168840 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:41.168840 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:41.168840 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:41.168840 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:41.168840 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:41.168840 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:21:41.168840 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:21:41.168840 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:21:41.168840 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:21:41.169931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:41.169931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:41.169931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:21:41.169931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:41.169931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:21:41.169931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:41.169931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:41.169931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:41.169931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:41.169931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:41.169931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:41.169931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:41.169931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:41.169931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:41.169931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:41.169931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:41.169931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:41.169931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:41.171078 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:41.171078 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:41.171078 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:21:41.171078 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:21:41.171078 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:21:41.171078 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:21:41.171078 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:21:41.171078 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:21:41.171078 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:21:41.171078 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:41.172428 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 366/728] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:21:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:41.173421 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f2607-4056-482f-9a6e-85d26e56d850 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:41.195656 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:41.208847 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc14939-b7a4-4133-aec2-dddad149a089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:41.209723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc14939-b7a4-4133-aec2-dddad149a089 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:41.210156 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc14939-b7a4-4133-aec2-dddad149a089 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:41.210614 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc14939-b7a4-4133-aec2-dddad149a089 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:41.211473 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc14939-b7a4-4133-aec2-dddad149a089 None admin] RBAC: Adding query filter params (name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:41.211961 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc14939-b7a4-4133-aec2-dddad149a089 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:41.213569 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc14939-b7a4-4133-aec2-dddad149a089 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:41.227729 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 363/729] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 1 16:21:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:41.229467 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc14939-b7a4-4133-aec2-dddad149a089 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:41.240363 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:41.282323 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f2607-4056-482f-9a6e-85d26e56d850 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:41.317308 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d62909-bb54-4fec-b513-8809f88f4d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:41.317936 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d62909-bb54-4fec-b513-8809f88f4d44 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:41.318477 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d62909-bb54-4fec-b513-8809f88f4d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:41.318600 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d62909-bb54-4fec-b513-8809f88f4d44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:41.319317 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d62909-bb54-4fec-b513-8809f88f4d44 None admin] RBAC: Adding query filter params (role.id=e8ba063905484d22993bcf04830091a3, scope.project.id=36ce80c4e0d64562a9ab3961be76fad9, user.id=278e6d6aed414c758fe3e9f4df6bf81e) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:41.319712 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d62909-bb54-4fec-b513-8809f88f4d44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:41.321066 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d62909-bb54-4fec-b513-8809f88f4d44 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:41.334576 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 367/730] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:21:41 2022] GET /identity/v3/role_assignments?user.id=278e6d6aed414c758fe3e9f4df6bf81e&scope.project.id=36ce80c4e0d64562a9ab3961be76fad9&role.id=e8ba063905484d22993bcf04830091a3 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:41.335701 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d62909-bb54-4fec-b513-8809f88f4d44 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:41.342930 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc14939-b7a4-4133-aec2-dddad149a089 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:41.385060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c0894-47cb-4fbb-8640-87feb48521c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:41.386023 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c0894-47cb-4fbb-8640-87feb48521c5 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:41.386520 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c0894-47cb-4fbb-8640-87feb48521c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:41.387017 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c0894-47cb-4fbb-8640-87feb48521c5 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:41.388060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7c0894-47cb-4fbb-8640-87feb48521c5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:41.388583 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7c0894-47cb-4fbb-8640-87feb48521c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:41.390336 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7c0894-47cb-4fbb-8640-87feb48521c5 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:41.411068 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 364/731] 10.222.0.18 () {56 vars in 1135 bytes} [Thu Sep 1 16:21:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:41.416426 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c0894-47cb-4fbb-8640-87feb48521c5 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:41.519719 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d62909-bb54-4fec-b513-8809f88f4d44 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:41.522622 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:41.556170 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:41.569660 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67017d70-6696-4ec7-9cb4-dc4918b86c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:41.571359 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67017d70-6696-4ec7-9cb4-dc4918b86c63 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:41.571739 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67017d70-6696-4ec7-9cb4-dc4918b86c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:41.572105 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67017d70-6696-4ec7-9cb4-dc4918b86c63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:41.587234 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67017d70-6696-4ec7-9cb4-dc4918b86c63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:41.589757 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67017d70-6696-4ec7-9cb4-dc4918b86c63 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:41.604972 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c0894-47cb-4fbb-8640-87feb48521c5 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:41.610029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-67017d70-6696-4ec7-9cb4-dc4918b86c63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:41.610029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:41.610029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:41.610029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:41.610029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:41.610029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:41.610029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:41.610029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:41.610029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:41.610029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:41.610029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:41.610029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:41.610029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:41.610029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:41.610029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:41.610029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:41.611178 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:41.611178 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:41.611178 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:41.611178 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:41.611178 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:41.611178 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:41.611178 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:41.611178 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:41.611178 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:41.611178 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:41.611178 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:41.611178 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:41.611178 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:41.611178 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:41.611178 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:41.611178 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:41.611178 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:41.612325 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:41.612325 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:41.612325 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:41.613648 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 368/732] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:41.619048 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea57390-5195-4511-ab09-6c7dbfdfb57e None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:41.621593 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea57390-5195-4511-ab09-6c7dbfdfb57e None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:41.622032 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea57390-5195-4511-ab09-6c7dbfdfb57e None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:41.623515 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 369/733] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:41 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:41.624098 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea57390-5195-4511-ab09-6c7dbfdfb57e None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:41.659961 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a4102-162d-4d9c-98dc-3f3ae2b17555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:41.661618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a4102-162d-4d9c-98dc-3f3ae2b17555 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:41.662225 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a4102-162d-4d9c-98dc-3f3ae2b17555 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:41.662801 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a4102-162d-4d9c-98dc-3f3ae2b17555 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:41.736623 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea57390-5195-4511-ab09-6c7dbfdfb57e None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:41.742024 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a4102-162d-4d9c-98dc-3f3ae2b17555 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:41.743588 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a4102-162d-4d9c-98dc-3f3ae2b17555 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:41.773002 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d8a8c-e6e8-4585-8840-9130d1857c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:41.773543 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d8a8c-e6e8-4585-8840-9130d1857c88 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:41.773825 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d8a8c-e6e8-4585-8840-9130d1857c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:41.774100 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d8a8c-e6e8-4585-8840-9130d1857c88 None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:41.774770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d8a8c-e6e8-4585-8840-9130d1857c88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:41.775078 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d8a8c-e6e8-4585-8840-9130d1857c88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:41.777021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d8a8c-e6e8-4585-8840-9130d1857c88 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:41.788960 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 370/734] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:41.790168 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48a441-6171-4a23-98bf-baa505d6b6e1 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:41.790487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48a441-6171-4a23-98bf-baa505d6b6e1 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:41.790765 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48a441-6171-4a23-98bf-baa505d6b6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:41.794888 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:41.808880 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-df48a441-6171-4a23-98bf-baa505d6b6e1 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:41.810075 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-df48a441-6171-4a23-98bf-baa505d6b6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:41.847475 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:41.862160 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 365/735] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:21:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:21:41.863470 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45719f-53fe-4b31-b30e-13d7afc81a2b None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:41.863763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45719f-53fe-4b31-b30e-13d7afc81a2b None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:41.864043 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45719f-53fe-4b31-b30e-13d7afc81a2b None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:41.869328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 366/736] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:41.870031 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b45719f-53fe-4b31-b30e-13d7afc81a2b None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:41.890279 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 371/737] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:21:41.890279 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48a441-6171-4a23-98bf-baa505d6b6e1 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:41.984875 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48a441-6171-4a23-98bf-baa505d6b6e1 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:42.012571 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b45719f-53fe-4b31-b30e-13d7afc81a2b None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:42.021649 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419858a5-9409-4ac5-a200-a47c0b81da2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:42.022309 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419858a5-9409-4ac5-a200-a47c0b81da2e None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:42.022589 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419858a5-9409-4ac5-a200-a47c0b81da2e None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:42.022859 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419858a5-9409-4ac5-a200-a47c0b81da2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:42.033254 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419858a5-9409-4ac5-a200-a47c0b81da2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:42.034741 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419858a5-9409-4ac5-a200-a47c0b81da2e None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:42.049699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-419858a5-9409-4ac5-a200-a47c0b81da2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:42.049699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:42.049699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:42.049699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:42.049699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:42.049699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:42.049699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:42.049699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:42.049699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:42.049699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:42.049699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:42.049699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:42.049699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:42.049699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:42.049699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:42.049699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:42.050659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:42.050659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:42.050659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:42.050659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:42.050659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:42.050659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:42.050659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:42.050659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:42.050659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:42.050659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:42.050659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:42.050659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:42.050659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:42.050659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:42.050659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:42.050659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:42.050659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:42.051514 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:42.051514 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:42.051514 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:42.051514 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 372/738] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:42.052222 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fedf0-1d3b-4e0e-b76e-4e59dd1f3c14 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:42.053352 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fedf0-1d3b-4e0e-b76e-4e59dd1f3c14 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:42.053352 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fedf0-1d3b-4e0e-b76e-4e59dd1f3c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:42.062483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1216a0d-9611-49b3-8484-2a1c14b6cb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:42.063390 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1216a0d-9611-49b3-8484-2a1c14b6cb96 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:42.063886 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1216a0d-9611-49b3-8484-2a1c14b6cb96 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:42.064361 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1216a0d-9611-49b3-8484-2a1c14b6cb96 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:42.072377 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-c96fedf0-1d3b-4e0e-b76e-4e59dd1f3c14 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:42.073951 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1216a0d-9611-49b3-8484-2a1c14b6cb96 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:42.074134 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-c96fedf0-1d3b-4e0e-b76e-4e59dd1f3c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:42.075789 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1216a0d-9611-49b3-8484-2a1c14b6cb96 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:42.091134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-a1216a0d-9611-49b3-8484-2a1c14b6cb96 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:21:42.091134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:42.091134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:42.091134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:42.091134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:42.091134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:42.091134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:42.091134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:42.091134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:42.091134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:42.091134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:42.091134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:42.091134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:21:42.091134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:21:42.091134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:21:42.091134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:21:42.092059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:42.092059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:42.092059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:21:42.092059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:42.092059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:21:42.092059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:42.092059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:42.092059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:42.092059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:42.092059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:42.092059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:42.092059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:42.092059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:42.092059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:42.092059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:42.092059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:42.092059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:42.092059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:42.092979 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:42.092979 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:42.092979 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:21:42.092979 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:21:42.092979 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:21:42.092979 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:21:42.092979 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:21:42.092979 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:21:42.092979 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:21:42.092979 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:42.094607 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:42.095806 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 367/739] 10.222.0.18 () {56 vars in 1117 bytes} [Thu Sep 1 16:21:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:42.096801 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1216a0d-9611-49b3-8484-2a1c14b6cb96 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:42.108025 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:42.153949 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 373/740] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:21:42.155234 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96fedf0-1d3b-4e0e-b76e-4e59dd1f3c14 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:42.219489 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1216a0d-9611-49b3-8484-2a1c14b6cb96 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:42.263869 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019d37f-2dfd-4077-bd12-29a6186bf34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:42.264829 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019d37f-2dfd-4077-bd12-29a6186bf34d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:42.265189 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019d37f-2dfd-4077-bd12-29a6186bf34d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:42.265883 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019d37f-2dfd-4077-bd12-29a6186bf34d None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:42.266853 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019d37f-2dfd-4077-bd12-29a6186bf34d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:42.267166 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019d37f-2dfd-4077-bd12-29a6186bf34d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:42.269220 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019d37f-2dfd-4077-bd12-29a6186bf34d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:42.271715 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96fedf0-1d3b-4e0e-b76e-4e59dd1f3c14 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:42.285328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 368/741] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:42.286914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019d37f-2dfd-4077-bd12-29a6186bf34d None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:42.313308 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13f5c9-9fcc-4b45-8f11-3f45b9e3baaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:42.314063 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13f5c9-9fcc-4b45-8f11-3f45b9e3baaa None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:42.314377 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13f5c9-9fcc-4b45-8f11-3f45b9e3baaa None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:42.314656 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13f5c9-9fcc-4b45-8f11-3f45b9e3baaa None admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:42.315436 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13f5c9-9fcc-4b45-8f11-3f45b9e3baaa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:42.315749 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13f5c9-9fcc-4b45-8f11-3f45b9e3baaa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:42.317424 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13f5c9-9fcc-4b45-8f11-3f45b9e3baaa None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:42.337310 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 374/742] 10.222.0.18 () {56 vars in 1149 bytes} [Thu Sep 1 16:21:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:42.338252 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13f5c9-9fcc-4b45-8f11-3f45b9e3baaa None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:42.353540 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:42.381629 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:42.435258 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019d37f-2dfd-4077-bd12-29a6186bf34d None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:42.458642 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13f5c9-9fcc-4b45-8f11-3f45b9e3baaa None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:42.478407 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5872e7e6-4154-4909-9c12-4aebc6e5fc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:42.479933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872e7e6-4154-4909-9c12-4aebc6e5fc89 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:42.480220 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872e7e6-4154-4909-9c12-4aebc6e5fc89 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:42.480497 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872e7e6-4154-4909-9c12-4aebc6e5fc89 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:42.481181 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5872e7e6-4154-4909-9c12-4aebc6e5fc89 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:42.483132 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5872e7e6-4154-4909-9c12-4aebc6e5fc89 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:42.499219 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448408f3-ad61-4466-a434-c95ff9d9c9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:42.499908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448408f3-ad61-4466-a434-c95ff9d9c9f2 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:42.500242 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448408f3-ad61-4466-a434-c95ff9d9c9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:42.500530 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448408f3-ad61-4466-a434-c95ff9d9c9f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:42.501088 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 369/743] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Sep 1 16:21:42 2022] POST /identity/v3/services => generated 251 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:21:42.501895 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5872e7e6-4154-4909-9c12-4aebc6e5fc89 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:42.527499 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448408f3-ad61-4466-a434-c95ff9d9c9f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:42.528990 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448408f3-ad61-4466-a434-c95ff9d9c9f2 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:42.547770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-448408f3-ad61-4466-a434-c95ff9d9c9f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:42.547770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:42.547770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:42.547770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:42.547770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:42.547770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:42.547770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:42.547770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:42.547770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:42.547770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:42.547770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:42.547770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:42.547770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:21:42.547770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:21:42.547770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:21:42.548747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:21:42.548747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:42.548747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:42.548747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:42.548747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:42.548747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:42.548747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:42.548747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:42.548747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:42.548747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:42.548747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:42.548747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:42.548747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:42.548747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:42.548747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:42.548747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:42.548747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:42.549609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:21:42.549609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:21:42.549609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:21:42.549609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:21:42.549609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:21:42.549609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:21:42.549609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:42.549609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:42.552619 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 375/744] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:42.553678 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448408f3-ad61-4466-a434-c95ff9d9c9f2 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:42.646552 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:42.651030 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5872e7e6-4154-4909-9c12-4aebc6e5fc89 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:42.671420 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:42.687216 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448408f3-ad61-4466-a434-c95ff9d9c9f2 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:42.696592 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c61fff-07dc-4d29-8054-0142897d1128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:42.697290 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c61fff-07dc-4d29-8054-0142897d1128 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:42.697673 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c61fff-07dc-4d29-8054-0142897d1128 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:42.698009 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c61fff-07dc-4d29-8054-0142897d1128 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:42.732484 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55951b9a-105d-401c-880f-7f35adca467b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:42.733183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55951b9a-105d-401c-880f-7f35adca467b None admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:42.733665 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55951b9a-105d-401c-880f-7f35adca467b None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:42.734006 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55951b9a-105d-401c-880f-7f35adca467b None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:42.735294 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55951b9a-105d-401c-880f-7f35adca467b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:42.736743 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55951b9a-105d-401c-880f-7f35adca467b None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:42.755757 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 376/745] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Sep 1 16:21:42 2022] POST /identity/v3/services => generated 240 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:21:42.756603 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55951b9a-105d-401c-880f-7f35adca467b None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:42.804844 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c61fff-07dc-4d29-8054-0142897d1128 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:42.807944 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c61fff-07dc-4d29-8054-0142897d1128 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:42.931314 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55951b9a-105d-401c-880f-7f35adca467b None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:42.968279 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:42.981236 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a440984-0302-47f0-be99-496df59f2840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:42.982144 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a440984-0302-47f0-be99-496df59f2840 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:42.982468 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a440984-0302-47f0-be99-496df59f2840 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:42.982748 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a440984-0302-47f0-be99-496df59f2840 None admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:42.983717 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a440984-0302-47f0-be99-496df59f2840 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:42.984031 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a440984-0302-47f0-be99-496df59f2840 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:42.987182 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a440984-0302-47f0-be99-496df59f2840 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:42.994195 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 370/746] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:21:42 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:21:42.996074 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c61fff-07dc-4d29-8054-0142897d1128 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:43.004154 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 377/747] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:21:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:43.010795 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a440984-0302-47f0-be99-496df59f2840 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:43.027018 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:43.154175 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c61fff-07dc-4d29-8054-0142897d1128 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:43.223122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b99a8-6e5b-4faf-b792-5175c252fe10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:43.223767 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b99a8-6e5b-4faf-b792-5175c252fe10 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:43.224019 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b99a8-6e5b-4faf-b792-5175c252fe10 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:43.224296 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b99a8-6e5b-4faf-b792-5175c252fe10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:43.243192 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b99a8-6e5b-4faf-b792-5175c252fe10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:43.245201 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b99a8-6e5b-4faf-b792-5175c252fe10 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:43.274262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-9f5b99a8-6e5b-4faf-b792-5175c252fe10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:43.274262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:43.274262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:43.274262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:43.274262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:43.274262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:43.274262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:43.274262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:43.274262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:43.274262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:43.274262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:43.274262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:43.274262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:43.274262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:43.274262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:43.274262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:43.275240 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:43.275240 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:43.275240 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:43.275240 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:43.275240 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:43.275240 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:43.275240 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:43.275240 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:43.275240 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:43.275240 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:43.275240 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:43.275240 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:43.275240 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:43.275240 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:43.275240 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:43.275240 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:43.275240 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:43.276134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:43.276134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:43.276134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:43.276134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 371/748] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:43.276969 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b99a8-6e5b-4faf-b792-5175c252fe10 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:43.320644 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a440984-0302-47f0-be99-496df59f2840 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:43.384840 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:43.386781 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb1908-616c-4e2f-9fd4-65f1fc1b6275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:43.387393 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb1908-616c-4e2f-9fd4-65f1fc1b6275 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:43.388495 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb1908-616c-4e2f-9fd4-65f1fc1b6275 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:43.388495 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb1908-616c-4e2f-9fd4-65f1fc1b6275 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:43.388878 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb1908-616c-4e2f-9fd4-65f1fc1b6275 None admin] RBAC: Adding query filter params (role.id=e8ba063905484d22993bcf04830091a3, scope.project.id=36ce80c4e0d64562a9ab3961be76fad9, user.id=8137f6b2ced040bd9752239c7e987f03) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:43.389184 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb1908-616c-4e2f-9fd4-65f1fc1b6275 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:43.390804 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb1908-616c-4e2f-9fd4-65f1fc1b6275 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:43.402361 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 378/749] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:21:43 2022] GET /identity/v3/role_assignments?user.id=8137f6b2ced040bd9752239c7e987f03&scope.project.id=36ce80c4e0d64562a9ab3961be76fad9&role.id=e8ba063905484d22993bcf04830091a3 => generated 259 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:21:43.445151 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e37cd-7c3e-4275-8920-15405f792b5d None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:43.445950 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e37cd-7c3e-4275-8920-15405f792b5d None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:43.446530 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e37cd-7c3e-4275-8920-15405f792b5d None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:43.448950 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 379/750] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:21:43.453051 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b99a8-6e5b-4faf-b792-5175c252fe10 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:43.454837 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610f3a5-1d4a-4b9f-b7d9-2a31020a38e2 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:43.458051 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610f3a5-1d4a-4b9f-b7d9-2a31020a38e2 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:43.458051 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610f3a5-1d4a-4b9f-b7d9-2a31020a38e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:43.475136 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-f610f3a5-1d4a-4b9f-b7d9-2a31020a38e2 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:43.476800 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-f610f3a5-1d4a-4b9f-b7d9-2a31020a38e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:43.532614 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96ad190-c42a-4663-97e4-ed60dcb65ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:43.533624 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ad190-c42a-4663-97e4-ed60dcb65ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:43.534706 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ad190-c42a-4663-97e4-ed60dcb65ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:43.535196 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ad190-c42a-4663-97e4-ed60dcb65ca7 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:43.536618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96ad190-c42a-4663-97e4-ed60dcb65ca7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:43.537539 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96ad190-c42a-4663-97e4-ed60dcb65ca7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:43.541391 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96ad190-c42a-4663-97e4-ed60dcb65ca7 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:43.567295 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 372/751] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:43.567932 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:43.571103 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96ad190-c42a-4663-97e4-ed60dcb65ca7 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:43.632431 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 380/752] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:21:43.677800 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96ad190-c42a-4663-97e4-ed60dcb65ca7 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:43.708140 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:43.715335 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961a7bc-801e-4cdd-bb3b-7cf1d460105e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:43.715972 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961a7bc-801e-4cdd-bb3b-7cf1d460105e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:43.716298 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961a7bc-801e-4cdd-bb3b-7cf1d460105e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:43.716570 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961a7bc-801e-4cdd-bb3b-7cf1d460105e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:43.723598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961a7bc-801e-4cdd-bb3b-7cf1d460105e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:43.725024 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961a7bc-801e-4cdd-bb3b-7cf1d460105e None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:43.736655 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-7961a7bc-801e-4cdd-bb3b-7cf1d460105e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:43.736655 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:43.736655 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:43.736655 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:43.736655 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:43.736655 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:43.736655 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:43.736655 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:43.736655 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:43.736655 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:43.736655 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:43.736655 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:43.736655 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:21:43.736655 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:21:43.736655 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:21:43.737510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:21:43.737510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:43.737510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:43.737510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:43.737510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:43.737510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:43.737510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:43.737510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:43.737510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:43.737510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:43.737510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:43.737510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:43.737510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:43.737510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:43.737510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:43.737510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:43.737510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:43.738676 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:21:43.738676 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:21:43.738676 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:21:43.738676 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:21:43.738676 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:21:43.738676 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:21:43.738676 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:43.738676 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:43.738676 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 373/753] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:43.743699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610f3a5-1d4a-4b9f-b7d9-2a31020a38e2 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:43.847916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610f3a5-1d4a-4b9f-b7d9-2a31020a38e2 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:43.891515 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45d11c-95cc-4dbb-9481-5587bcff464d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:43.892502 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45d11c-95cc-4dbb-9481-5587bcff464d None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:43.892957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45d11c-95cc-4dbb-9481-5587bcff464d None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:43.893399 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45d11c-95cc-4dbb-9481-5587bcff464d None admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:43.894319 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd45d11c-95cc-4dbb-9481-5587bcff464d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:43.894800 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd45d11c-95cc-4dbb-9481-5587bcff464d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:43.896413 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd45d11c-95cc-4dbb-9481-5587bcff464d None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:43.912917 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 381/754] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:21:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:43.919516 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961a7bc-801e-4cdd-bb3b-7cf1d460105e None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:44.030680 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961a7bc-801e-4cdd-bb3b-7cf1d460105e None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:44.059791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45d11c-95cc-4dbb-9481-5587bcff464d None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:44.091510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa3ab6-4454-4218-bb31-8493a6b8f73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:44.092391 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa3ab6-4454-4218-bb31-8493a6b8f73b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:44.092844 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa3ab6-4454-4218-bb31-8493a6b8f73b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:44.093288 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa3ab6-4454-4218-bb31-8493a6b8f73b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:44.094181 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaa3ab6-4454-4218-bb31-8493a6b8f73b None admin] RBAC: Adding query filter params (role.id=b6727ed962fa41109b0a2d6851551f49, scope.project.id=36ce80c4e0d64562a9ab3961be76fad9, user.id=a5b3e53fd1314e17ad28c42a44e8eff3) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:44.094703 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaa3ab6-4454-4218-bb31-8493a6b8f73b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:44.096351 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaa3ab6-4454-4218-bb31-8493a6b8f73b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:44.103322 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 374/755] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:21:43 2022] GET /identity/v3/role_assignments?user.id=a5b3e53fd1314e17ad28c42a44e8eff3&scope.project.id=36ce80c4e0d64562a9ab3961be76fad9&role.id=b6727ed962fa41109b0a2d6851551f49 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:44.145691 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:44.221707 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45d11c-95cc-4dbb-9481-5587bcff464d None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:44.258925 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64caec95-7e74-4cdb-8486-9e846d6ae3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:44.259617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64caec95-7e74-4cdb-8486-9e846d6ae3ff None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:44.259947 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64caec95-7e74-4cdb-8486-9e846d6ae3ff None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:44.260226 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64caec95-7e74-4cdb-8486-9e846d6ae3ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:44.278192 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64caec95-7e74-4cdb-8486-9e846d6ae3ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:44.287321 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-64caec95-7e74-4cdb-8486-9e846d6ae3ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:21:44.287321 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:44.287321 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:44.287321 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:44.287321 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:44.287321 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:44.287321 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:44.287321 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:44.287321 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:44.287321 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:44.287321 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:44.287321 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:44.287321 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:44.287321 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:44.287321 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:44.287321 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:44.288259 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:44.288259 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:44.288259 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:44.288259 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:44.288259 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:44.288259 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:44.288259 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:44.288259 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:44.288259 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:44.288259 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:44.288259 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:44.288259 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:44.288259 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:44.288259 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:44.288259 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:44.288259 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:44.289135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:44.289135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:44.289135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:44.289135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:44.289135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:44.289135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:44.289135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:44.289135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:44.289135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:44.289135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:44.289135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:21:44.289135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:44.289135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 382/756] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:21:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:44.295142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa3ab6-4454-4218-bb31-8493a6b8f73b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:44.370258 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:44.436991 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa3ab6-4454-4218-bb31-8493a6b8f73b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:44.482504 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d258388-b41d-40ab-96d8-6cfdcadb4c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:44.483125 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d258388-b41d-40ab-96d8-6cfdcadb4c74 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:44.483408 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d258388-b41d-40ab-96d8-6cfdcadb4c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:44.483683 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d258388-b41d-40ab-96d8-6cfdcadb4c74 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:44.484349 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d258388-b41d-40ab-96d8-6cfdcadb4c74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:44.484654 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d258388-b41d-40ab-96d8-6cfdcadb4c74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:44.486020 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d258388-b41d-40ab-96d8-6cfdcadb4c74 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:44.495798 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 375/757] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:21:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:44.501304 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64caec95-7e74-4cdb-8486-9e846d6ae3ff None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:44.603332 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:44.606854 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64caec95-7e74-4cdb-8486-9e846d6ae3ff None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:44.641922 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5411975-d141-4d57-bb95-c2e63bf9b80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:44.642579 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5411975-d141-4d57-bb95-c2e63bf9b80f None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:44.642856 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5411975-d141-4d57-bb95-c2e63bf9b80f None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:44.643128 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5411975-d141-4d57-bb95-c2e63bf9b80f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:44.653992 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5411975-d141-4d57-bb95-c2e63bf9b80f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:44.655471 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5411975-d141-4d57-bb95-c2e63bf9b80f None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:44.674013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-a5411975-d141-4d57-bb95-c2e63bf9b80f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:44.674013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:44.674013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:44.674013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:44.674013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:44.674013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:44.674013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:44.674013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:44.674013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:44.674013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:44.674013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:44.674013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:44.674013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:44.674013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:44.674013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:44.674013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:44.674995 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:44.674995 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:44.674995 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:44.674995 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:44.674995 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:44.674995 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:44.674995 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:44.674995 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:44.674995 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:44.674995 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:44.674995 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:44.674995 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:44.674995 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:44.674995 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:44.674995 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:44.674995 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:44.674995 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:44.675851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:44.675851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:44.675851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:44.675851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 383/758] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:44.683151 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d258388-b41d-40ab-96d8-6cfdcadb4c74 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:44.817133 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d258388-b41d-40ab-96d8-6cfdcadb4c74 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:44.836853 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:44.862061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f579c4c-14c5-4f4c-b234-4eb59032ad72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:44.862634 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f579c4c-14c5-4f4c-b234-4eb59032ad72 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:44.863009 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f579c4c-14c5-4f4c-b234-4eb59032ad72 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:44.863281 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f579c4c-14c5-4f4c-b234-4eb59032ad72 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:44.863926 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f579c4c-14c5-4f4c-b234-4eb59032ad72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:44.864356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f579c4c-14c5-4f4c-b234-4eb59032ad72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:44.865710 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f579c4c-14c5-4f4c-b234-4eb59032ad72 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:44.884251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 376/759] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:44.890261 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5411975-d141-4d57-bb95-c2e63bf9b80f None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:45.001414 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed6853-4ce7-4e6b-add7-23ebfd99dc6f None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:45.001741 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed6853-4ce7-4e6b-add7-23ebfd99dc6f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:45.001969 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed6853-4ce7-4e6b-add7-23ebfd99dc6f None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:45.003615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 377/760] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:45.013702 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636fe61-4830-4691-bc13-afcd43c79a69 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:45.014002 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636fe61-4830-4691-bc13-afcd43c79a69 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:45.014313 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636fe61-4830-4691-bc13-afcd43c79a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:45.029815 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5411975-d141-4d57-bb95-c2e63bf9b80f None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:45.050262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-8636fe61-4830-4691-bc13-afcd43c79a69 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:45.051527 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8636fe61-4830-4691-bc13-afcd43c79a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:45.089922 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdb068-9789-4ec0-af1f-a1480f0d7c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:45.090578 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdb068-9789-4ec0-af1f-a1480f0d7c0e None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:45.090856 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdb068-9789-4ec0-af1f-a1480f0d7c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:45.091126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdb068-9789-4ec0-af1f-a1480f0d7c0e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:45.103954 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdb068-9789-4ec0-af1f-a1480f0d7c0e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:45.105385 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdb068-9789-4ec0-af1f-a1480f0d7c0e None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:45.120157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-b4fdb068-9789-4ec0-af1f-a1480f0d7c0e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:21:45.120157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:45.120157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:45.120157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:45.120157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:45.120157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:45.120157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:45.120157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:45.120157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:45.120157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:45.120157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:45.120157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:45.120157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:21:45.120157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:21:45.120157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:21:45.120157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:21:45.121380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:45.121380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:45.121380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:21:45.121380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:45.121380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:21:45.121380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:45.121380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:45.121380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:45.121380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:45.121380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:45.121380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:45.121380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:45.121380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:45.121380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:45.121380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:45.121380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:45.121380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:45.121380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:45.122344 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:45.122344 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:45.122344 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:21:45.122344 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:21:45.122344 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:21:45.122344 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:21:45.122344 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:21:45.122344 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:21:45.122344 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:21:45.122344 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:45.122344 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 384/761] 10.222.0.18 () {56 vars in 1093 bytes} [Thu Sep 1 16:21:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:45.127331 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdb068-9789-4ec0-af1f-a1480f0d7c0e None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:45.146879 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:45.208030 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 378/762] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:21:45.245727 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdb068-9789-4ec0-af1f-a1480f0d7c0e None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:45.291213 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b75b887-b6cf-4dcd-b035-cfa5d4233bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:45.291832 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75b887-b6cf-4dcd-b035-cfa5d4233bfc None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:45.292109 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75b887-b6cf-4dcd-b035-cfa5d4233bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:45.292380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75b887-b6cf-4dcd-b035-cfa5d4233bfc None admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:45.293103 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b75b887-b6cf-4dcd-b035-cfa5d4233bfc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:45.293410 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b75b887-b6cf-4dcd-b035-cfa5d4233bfc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:45.294868 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b75b887-b6cf-4dcd-b035-cfa5d4233bfc None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:45.324550 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 385/763] 10.222.0.18 () {56 vars in 1133 bytes} [Thu Sep 1 16:21:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:45.330270 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8636fe61-4830-4691-bc13-afcd43c79a69 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:45.473431 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8636fe61-4830-4691-bc13-afcd43c79a69 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:45.484896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:45.521268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c34eaa-c4c5-4e36-917e-d0982ef8791a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:45.521909 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c34eaa-c4c5-4e36-917e-d0982ef8791a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:45.522188 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c34eaa-c4c5-4e36-917e-d0982ef8791a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:45.522487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c34eaa-c4c5-4e36-917e-d0982ef8791a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:45.537886 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c34eaa-c4c5-4e36-917e-d0982ef8791a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:45.539362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c34eaa-c4c5-4e36-917e-d0982ef8791a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:45.550149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88b761-a65d-406d-af33-29d2ca264fcd None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:45.550471 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88b761-a65d-406d-af33-29d2ca264fcd None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:45.550755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88b761-a65d-406d-af33-29d2ca264fcd None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:45.552440 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 386/764] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:45.557884 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3bccf-6014-434e-b6ac-ef9c9fa57969 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:45.558167 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3bccf-6014-434e-b6ac-ef9c9fa57969 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:45.558468 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3bccf-6014-434e-b6ac-ef9c9fa57969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:45.562928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-32c34eaa-c4c5-4e36-917e-d0982ef8791a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:45.562928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:45.562928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:45.562928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:45.562928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:45.562928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:45.562928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:45.562928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:45.562928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:45.562928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:45.562928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:45.562928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:45.562928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:45.562928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:45.562928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:45.562928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:45.563805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:45.563805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:45.563805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:45.563805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:45.563805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:45.563805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:45.563805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:45.563805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:45.563805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:45.563805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:45.563805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:45.563805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:45.563805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:45.563805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:45.563805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:45.563805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:45.563805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:45.564664 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:45.564664 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:45.564664 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:45.564664 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 379/765] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:45.569533 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c34eaa-c4c5-4e36-917e-d0982ef8791a None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:45.586499 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-4af3bccf-6014-434e-b6ac-ef9c9fa57969 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:45.588386 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-4af3bccf-6014-434e-b6ac-ef9c9fa57969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:45.672020 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:45.677449 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 387/766] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:21:45.678725 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3ff2c-4865-4141-a233-bdc15ee04613 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:45.679004 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3ff2c-4865-4141-a233-bdc15ee04613 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:45.679270 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3ff2c-4865-4141-a233-bdc15ee04613 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:45.685125 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 388/767] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:45.685652 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a3ff2c-4865-4141-a233-bdc15ee04613 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:45.705373 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c34eaa-c4c5-4e36-917e-d0982ef8791a None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:45.754346 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de1070-d8fd-4658-8bee-852ab0d44a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:45.755565 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de1070-d8fd-4658-8bee-852ab0d44a75 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:45.756274 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de1070-d8fd-4658-8bee-852ab0d44a75 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:45.756729 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de1070-d8fd-4658-8bee-852ab0d44a75 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:45.758133 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de1070-d8fd-4658-8bee-852ab0d44a75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:45.758652 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de1070-d8fd-4658-8bee-852ab0d44a75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:45.760783 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de1070-d8fd-4658-8bee-852ab0d44a75 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:45.777352 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 380/768] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:21:45.779284 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d1d01-3759-41d2-9a2b-b111a4a84e91 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:45.779573 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d1d01-3759-41d2-9a2b-b111a4a84e91 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:45.779874 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d1d01-3759-41d2-9a2b-b111a4a84e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:45.798632 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-493d1d01-3759-41d2-9a2b-b111a4a84e91 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:45.799863 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-493d1d01-3759-41d2-9a2b-b111a4a84e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:45.804292 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a3ff2c-4865-4141-a233-bdc15ee04613 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:45.814381 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:45.849558 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d0a5c-9fc0-4dfe-ac80-5bfedd267e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:45.851149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d0a5c-9fc0-4dfe-ac80-5bfedd267e60 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:45.851634 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d0a5c-9fc0-4dfe-ac80-5bfedd267e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:45.852253 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d0a5c-9fc0-4dfe-ac80-5bfedd267e60 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:45.853401 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6d0a5c-9fc0-4dfe-ac80-5bfedd267e60 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:45.855269 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6d0a5c-9fc0-4dfe-ac80-5bfedd267e60 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:45.868257 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-1a6d0a5c-9fc0-4dfe-ac80-5bfedd267e60 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:21:45.868257 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:45.868257 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:21:45.868257 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:21:45.868257 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:21:45.868257 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:21:45.868257 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:21:45.868257 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:21:45.868257 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:21:45.868257 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:45.868257 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:21:45.868257 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:45.868257 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:45.868257 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:45.868257 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:45.868257 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:45.868257 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:45.869767 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:45.869767 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:45.869767 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:45.869767 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:45.869767 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:45.869767 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:45.869767 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:21:45.869767 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:21:45.869767 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:21:45.869767 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:21:45.869767 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:45.869767 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:45.869767 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:45.869767 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:45.869767 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:45.869767 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:45.870876 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:45.870876 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:45.870876 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:45.870876 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:45.870876 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:45.870876 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:45.870876 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:45.870876 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:45.870876 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:45.870876 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:45.870876 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:21:45.870876 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:21:45.870876 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:21:45.870876 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:45.873541 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 389/769] 10.222.0.18 () {56 vars in 1129 bytes} [Thu Sep 1 16:21:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:45.874716 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d0a5c-9fc0-4dfe-ac80-5bfedd267e60 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:45.882608 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 381/770] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:21:45.883608 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d1d01-3759-41d2-9a2b-b111a4a84e91 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:45.955335 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:46.011072 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d0a5c-9fc0-4dfe-ac80-5bfedd267e60 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:46.034669 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d1d01-3759-41d2-9a2b-b111a4a84e91 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:46.053626 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377905c8-6f89-49ab-8af6-4a866094daeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:46.054586 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377905c8-6f89-49ab-8af6-4a866094daeb None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:46.055171 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377905c8-6f89-49ab-8af6-4a866094daeb None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:46.055635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377905c8-6f89-49ab-8af6-4a866094daeb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:46.065052 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377905c8-6f89-49ab-8af6-4a866094daeb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:46.066900 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377905c8-6f89-49ab-8af6-4a866094daeb None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:46.079658 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec28e011-47c8-4a2a-8386-99d4b333c6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:46.080399 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec28e011-47c8-4a2a-8386-99d4b333c6af None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:46.080778 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec28e011-47c8-4a2a-8386-99d4b333c6af None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:46.081120 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec28e011-47c8-4a2a-8386-99d4b333c6af None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:46.081825 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec28e011-47c8-4a2a-8386-99d4b333c6af None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:46.082203 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec28e011-47c8-4a2a-8386-99d4b333c6af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:46.082606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-377905c8-6f89-49ab-8af6-4a866094daeb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:46.082606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:46.082606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:46.082606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:46.082606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:46.082606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:46.082606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:46.082606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:46.082606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:46.082606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:46.082606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:46.082606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:46.082606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:21:46.082606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:21:46.082606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:21:46.083441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:21:46.083441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:46.083441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:46.083441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:46.083441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:46.083441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:46.083441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:46.083441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:46.083441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:46.083441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:46.083441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:46.083441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:46.083441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:46.083441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:46.083441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:46.083441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:46.083441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:46.084298 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:21:46.084298 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:21:46.084298 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:21:46.084298 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:21:46.084298 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:21:46.084298 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:21:46.084298 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:46.084298 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:46.084713 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec28e011-47c8-4a2a-8386-99d4b333c6af None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:46.086771 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 390/771] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:46.087812 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377905c8-6f89-49ab-8af6-4a866094daeb None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:46.095205 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 382/772] 10.222.0.18 () {56 vars in 1124 bytes} [Thu Sep 1 16:21:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:46.096186 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec28e011-47c8-4a2a-8386-99d4b333c6af None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:46.114516 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:46.215397 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377905c8-6f89-49ab-8af6-4a866094daeb None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:46.227226 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec28e011-47c8-4a2a-8386-99d4b333c6af None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:46.249839 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:46.262070 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a6b81-4391-42c6-8071-e382b39754f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:46.262684 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a6b81-4391-42c6-8071-e382b39754f2 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:46.262968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a6b81-4391-42c6-8071-e382b39754f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:46.263242 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a6b81-4391-42c6-8071-e382b39754f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:46.264078 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f17420-9a7d-481a-a9a5-573db16e2f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:46.264695 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f17420-9a7d-481a-a9a5-573db16e2f9c None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:46.264987 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f17420-9a7d-481a-a9a5-573db16e2f9c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:46.265265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f17420-9a7d-481a-a9a5-573db16e2f9c None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:46.265968 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f17420-9a7d-481a-a9a5-573db16e2f9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:46.266323 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f17420-9a7d-481a-a9a5-573db16e2f9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:46.267760 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f17420-9a7d-481a-a9a5-573db16e2f9c None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:46.274051 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9a6b81-4391-42c6-8071-e382b39754f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:46.279264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-ec9a6b81-4391-42c6-8071-e382b39754f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:21:46.279264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:46.279264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:46.279264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:46.279264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:46.279264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:46.279264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:46.279264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:46.279264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:46.279264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:46.279264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:46.279264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:46.279264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:46.279264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:46.279264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:46.279264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:46.280342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:46.280342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:46.280342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:46.280342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:46.280342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:46.280342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:46.280342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:46.280342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:46.280342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:46.280342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:46.280342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:46.280342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:46.280342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:46.280342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:46.280342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:46.280342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:46.281176 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:46.281176 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:46.281176 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:46.281176 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:46.281176 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:46.281176 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:46.281176 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:46.281176 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:46.281176 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:46.281176 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:46.281176 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:21:46.281176 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:46.281176 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 391/773] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:21:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:46.281855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a6b81-4391-42c6-8071-e382b39754f2 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:46.284353 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 383/774] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:21:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:46.285908 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f17420-9a7d-481a-a9a5-573db16e2f9c None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:46.354535 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:46.383225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a6b81-4391-42c6-8071-e382b39754f2 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:46.387094 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f17420-9a7d-481a-a9a5-573db16e2f9c None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:46.415947 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904303a8-da9a-4a0d-a83f-8f0d3f914af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:46.417066 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904303a8-da9a-4a0d-a83f-8f0d3f914af7 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:46.417350 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904303a8-da9a-4a0d-a83f-8f0d3f914af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:46.417624 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904303a8-da9a-4a0d-a83f-8f0d3f914af7 None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:46.418266 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904303a8-da9a-4a0d-a83f-8f0d3f914af7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:46.418583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904303a8-da9a-4a0d-a83f-8f0d3f914af7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:46.419961 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904303a8-da9a-4a0d-a83f-8f0d3f914af7 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:46.420138 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f0de5-ba61-4da2-b503-74fe02561a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:46.420759 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f0de5-ba61-4da2-b503-74fe02561a0c None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:46.421042 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f0de5-ba61-4da2-b503-74fe02561a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:46.421312 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f0de5-ba61-4da2-b503-74fe02561a0c None admin] PATH_INFO: `/v3/services/network` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:46.421889 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f0de5-ba61-4da2-b503-74fe02561a0c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:46.423249 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f0de5-ba61-4da2-b503-74fe02561a0c None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:46.424797 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 392/775] 10.222.0.18 () {56 vars in 1124 bytes} [Thu Sep 1 16:21:46 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:46.424797 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a5b5c-d929-41f0-b588-7f41c97cd104 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:46.424797 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a5b5c-d929-41f0-b588-7f41c97cd104 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:46.424797 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a5b5c-d929-41f0-b588-7f41c97cd104 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:46.424797 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 393/776] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:46.424797 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a5b5c-d929-41f0-b588-7f41c97cd104 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:46.439382 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-689f0de5-ba61-4da2-b503-74fe02561a0c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:21:46.439382 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:46.439382 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:21:46.439382 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:21:46.439382 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:21:46.439382 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:21:46.439382 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:21:46.439382 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:21:46.439382 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:21:46.439382 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:46.439382 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:21:46.439382 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:46.439382 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:46.439382 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:46.439382 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:46.439382 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:46.439382 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:46.440321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:46.440321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:46.440321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:46.440321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:46.440321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:46.440321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:46.440321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:21:46.440321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:21:46.440321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:21:46.440321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:21:46.440321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:46.440321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:46.440321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:46.440321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:46.440321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:46.440321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:46.441254 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:46.441254 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:46.441254 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:46.441254 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:46.441254 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:46.441254 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:46.441254 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:46.441254 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:46.441254 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:46.441254 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:46.441254 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:21:46.441254 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:21:46.441254 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:21:46.441254 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:46.441254 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 384/777] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:46 2022] GET /identity/v3/services/network => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:46.442015 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f0de5-ba61-4da2-b503-74fe02561a0c None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:46.480354 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:46.532567 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a5b5c-d929-41f0-b588-7f41c97cd104 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:46.540743 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f0de5-ba61-4da2-b503-74fe02561a0c None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:46.565795 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11d315-971b-4006-a039-910e7c9e71ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:46.566495 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11d315-971b-4006-a039-910e7c9e71ad None admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:46.566778 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11d315-971b-4006-a039-910e7c9e71ad None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:46.567051 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11d315-971b-4006-a039-910e7c9e71ad None admin] PATH_INFO: `/v3/projects/36ce80c4e0d64562a9ab3961be76fad9/users/278e6d6aed414c758fe3e9f4df6bf81e/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:46.568945 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11d315-971b-4006-a039-910e7c9e71ad None admin] RBAC: Authorizing `identity:create_grant(project_id=36ce80c4e0d64562a9ab3961be76fad9, user_id=278e6d6aed414c758fe3e9f4df6bf81e, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:46.570412 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11d315-971b-4006-a039-910e7c9e71ad None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:46.573822 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6918e8-5cab-42ef-91da-fdade4f481bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:46.574542 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6918e8-5cab-42ef-91da-fdade4f481bf None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:46.574958 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6918e8-5cab-42ef-91da-fdade4f481bf None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:46.575245 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6918e8-5cab-42ef-91da-fdade4f481bf None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:46.576032 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6918e8-5cab-42ef-91da-fdade4f481bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:46.576414 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6918e8-5cab-42ef-91da-fdade4f481bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:46.577708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6918e8-5cab-42ef-91da-fdade4f481bf None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:46.580548 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:46.582899 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 394/778] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 1 16:21:46 2022] PUT /identity/v3/projects/36ce80c4e0d64562a9ab3961be76fad9/users/278e6d6aed414c758fe3e9f4df6bf81e/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:21:46.583717 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11d315-971b-4006-a039-910e7c9e71ad None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:46.599158 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 385/779] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:21:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:46.600501 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7834f-8f02-49f7-a084-fa5f2811cc7e None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:46.600784 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7834f-8f02-49f7-a084-fa5f2811cc7e None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:46.601067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7834f-8f02-49f7-a084-fa5f2811cc7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:46.622794 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-a7e7834f-8f02-49f7-a084-fa5f2811cc7e None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:46.624215 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a7e7834f-8f02-49f7-a084-fa5f2811cc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:46.706510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11d315-971b-4006-a039-910e7c9e71ad None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:46.707051 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 386/780] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:21:46.707051 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e7834f-8f02-49f7-a084-fa5f2811cc7e None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:46.717682 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:46.754761 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37a7d6-acac-4c77-ace2-12c99ede51d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:46.755624 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37a7d6-acac-4c77-ace2-12c99ede51d8 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:46.756076 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37a7d6-acac-4c77-ace2-12c99ede51d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:46.756491 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37a7d6-acac-4c77-ace2-12c99ede51d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:46.757328 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a37a7d6-acac-4c77-ace2-12c99ede51d8 None admin] RBAC: Adding query filter params (interface=public, service_id=8e53e1b0ee8749ea8cba6145ee38c78b, region_id=RegionOne) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:46.757790 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a37a7d6-acac-4c77-ace2-12c99ede51d8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:46.759875 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a37a7d6-acac-4c77-ace2-12c99ede51d8 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:46.769804 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 395/781] 10.222.0.18 () {56 vars in 1251 bytes} [Thu Sep 1 16:21:46 2022] GET /identity/v3/endpoints?service_id=8e53e1b0ee8749ea8cba6145ee38c78b&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:46.770688 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37a7d6-acac-4c77-ace2-12c99ede51d8 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:46.824041 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e7834f-8f02-49f7-a084-fa5f2811cc7e None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:46.851852 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:46.862948 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdfeaa3-13bc-4cff-85d7-0efa9b81ffc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:46.863812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdfeaa3-13bc-4cff-85d7-0efa9b81ffc8 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:46.864236 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdfeaa3-13bc-4cff-85d7-0efa9b81ffc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:46.864741 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdfeaa3-13bc-4cff-85d7-0efa9b81ffc8 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:46.865683 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdfeaa3-13bc-4cff-85d7-0efa9b81ffc8 None admin] RBAC: Adding query filter params (name=network) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:46.866131 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdfeaa3-13bc-4cff-85d7-0efa9b81ffc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:46.867689 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdfeaa3-13bc-4cff-85d7-0efa9b81ffc8 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:46.873105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 387/782] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Sep 1 16:21:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:46.874229 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdfeaa3-13bc-4cff-85d7-0efa9b81ffc8 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:46.897376 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37a7d6-acac-4c77-ace2-12c99ede51d8 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:46.933051 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4412e-aaba-4a35-a016-9280a9f28767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:46.933691 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4412e-aaba-4a35-a016-9280a9f28767 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:46.933974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4412e-aaba-4a35-a016-9280a9f28767 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:46.934261 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4412e-aaba-4a35-a016-9280a9f28767 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:46.945621 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4412e-aaba-4a35-a016-9280a9f28767 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:46.947134 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4412e-aaba-4a35-a016-9280a9f28767 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:46.962620 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-77f4412e-aaba-4a35-a016-9280a9f28767 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:46.962620 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:46.962620 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:46.962620 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:46.962620 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:46.962620 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:46.962620 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:46.962620 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:46.962620 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:46.962620 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:46.962620 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:46.962620 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:46.962620 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:46.962620 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:46.962620 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:46.962620 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:46.964243 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:46.964243 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:46.964243 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:46.964243 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:46.964243 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:46.964243 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:46.964243 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:46.964243 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:46.964243 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:46.964243 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:46.964243 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:46.964243 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:46.964243 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:46.964243 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:46.964243 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:46.964243 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:46.964243 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:46.965733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:46.965733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:46.965733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:46.965733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 396/783] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:46.965733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4412e-aaba-4a35-a016-9280a9f28767 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:46.988510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdfeaa3-13bc-4cff-85d7-0efa9b81ffc8 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:46.992794 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:47.023307 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1355a-c3c8-4c20-a1d9-ff9570a2b898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:47.023921 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1355a-c3c8-4c20-a1d9-ff9570a2b898 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:47.024206 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1355a-c3c8-4c20-a1d9-ff9570a2b898 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:47.024483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1355a-c3c8-4c20-a1d9-ff9570a2b898 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:47.025099 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1355a-c3c8-4c20-a1d9-ff9570a2b898 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:47.027064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1355a-c3c8-4c20-a1d9-ff9570a2b898 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:47.042267 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 388/784] 10.222.0.18 () {56 vars in 1087 bytes} [Thu Sep 1 16:21:46 2022] GET /identity/v3/services => generated 772 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:47.042267 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1355a-c3c8-4c20-a1d9-ff9570a2b898 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:47.096682 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4412e-aaba-4a35-a016-9280a9f28767 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:47.121983 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:47.136737 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4220cb3-5f94-4fbd-a8c2-67a75048518c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:47.137487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4220cb3-5f94-4fbd-a8c2-67a75048518c None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:47.137832 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4220cb3-5f94-4fbd-a8c2-67a75048518c None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:47.138168 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4220cb3-5f94-4fbd-a8c2-67a75048518c None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:47.138980 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4220cb3-5f94-4fbd-a8c2-67a75048518c None admin] RBAC: Adding query filter params (type=network) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:47.139346 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4220cb3-5f94-4fbd-a8c2-67a75048518c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:47.140959 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4220cb3-5f94-4fbd-a8c2-67a75048518c None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:47.147069 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 397/785] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Sep 1 16:21:46 2022] GET /identity/v3/services?type=network => generated 350 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:47.148109 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4220cb3-5f94-4fbd-a8c2-67a75048518c None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:47.175268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1355a-c3c8-4c20-a1d9-ff9570a2b898 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:47.217872 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d2f346-ef71-4af4-8a8a-5881f9b98387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:47.218086 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2f346-ef71-4af4-8a8a-5881f9b98387 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:47.218371 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2f346-ef71-4af4-8a8a-5881f9b98387 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:47.218552 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2f346-ef71-4af4-8a8a-5881f9b98387 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:47.219188 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d2f346-ef71-4af4-8a8a-5881f9b98387 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:47.219488 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d2f346-ef71-4af4-8a8a-5881f9b98387 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:47.220846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d2f346-ef71-4af4-8a8a-5881f9b98387 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:47.233327 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 389/786] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:47.234180 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d2f346-ef71-4af4-8a8a-5881f9b98387 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:47.256572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:47.278575 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4220cb3-5f94-4fbd-a8c2-67a75048518c None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:47.315515 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ccd45-7f06-4d04-ac34-6ff0e359d456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:47.316145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ccd45-7f06-4d04-ac34-6ff0e359d456 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:47.316420 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ccd45-7f06-4d04-ac34-6ff0e359d456 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:47.316740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ccd45-7f06-4d04-ac34-6ff0e359d456 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:47.327948 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3ccd45-7f06-4d04-ac34-6ff0e359d456 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:47.332769 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-8e3ccd45-7f06-4d04-ac34-6ff0e359d456 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:47.332769 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:47.332769 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:47.332769 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:47.332769 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:47.332769 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:47.332769 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:47.332769 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:47.332769 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:47.332769 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:47.332769 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:47.332769 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:47.332769 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:47.332769 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:47.332769 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:47.332769 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:47.333686 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:47.333686 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:47.333686 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:47.333686 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:47.333686 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:47.333686 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:47.333686 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:47.333686 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:47.333686 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:47.333686 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:47.333686 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:47.333686 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:47.333686 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:47.333686 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:47.333686 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:47.333686 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:47.334535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:47.334535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:47.334535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:47.334535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:47.334535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:47.334535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:47.334535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:47.334535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:47.334535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:47.334535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:47.334535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:47.334535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:47.334535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 398/787] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 1 16:21:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:47.335228 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ccd45-7f06-4d04-ac34-6ff0e359d456 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:47.378991 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d2f346-ef71-4af4-8a8a-5881f9b98387 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:47.397144 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:47.418101 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f021a-3df3-41c6-9ba9-de7821851d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:47.419070 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f021a-3df3-41c6-9ba9-de7821851d9f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:47.419585 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f021a-3df3-41c6-9ba9-de7821851d9f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:47.420074 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f021a-3df3-41c6-9ba9-de7821851d9f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:47.420996 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f021a-3df3-41c6-9ba9-de7821851d9f None admin] RBAC: Adding query filter params (interface=public, service_id=fc19123823cd42fbbc3ef27bc7a4ea70, region_id=RegionOne) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:47.421505 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f021a-3df3-41c6-9ba9-de7821851d9f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:47.423101 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f021a-3df3-41c6-9ba9-de7821851d9f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:47.436188 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 390/788] 10.222.0.18 () {56 vars in 1251 bytes} [Thu Sep 1 16:21:47 2022] GET /identity/v3/endpoints?service_id=fc19123823cd42fbbc3ef27bc7a4ea70&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:47.437362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f021a-3df3-41c6-9ba9-de7821851d9f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:47.446951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ccd45-7f06-4d04-ac34-6ff0e359d456 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:47.483336 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d93ce20-6cb6-473a-b66e-046f85a6a130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:47.483980 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93ce20-6cb6-473a-b66e-046f85a6a130 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:47.484260 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93ce20-6cb6-473a-b66e-046f85a6a130 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:47.484535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93ce20-6cb6-473a-b66e-046f85a6a130 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:47.493571 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d93ce20-6cb6-473a-b66e-046f85a6a130 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:47.496566 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d93ce20-6cb6-473a-b66e-046f85a6a130 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:47.514464 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-6d93ce20-6cb6-473a-b66e-046f85a6a130 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:21:47.514464 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:47.514464 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:47.514464 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:47.514464 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:47.514464 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:47.514464 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:47.514464 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:47.514464 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:47.514464 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:47.514464 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:47.514464 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:47.514464 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:21:47.514464 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:21:47.514464 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:21:47.514464 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:21:47.515436 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:47.515436 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:47.515436 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:21:47.515436 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:47.515436 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:21:47.515436 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:47.515436 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:47.515436 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:47.515436 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:47.515436 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:47.515436 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:47.515436 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:47.515436 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:47.515436 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:47.515436 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:47.515436 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:47.515436 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:47.515436 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:47.516366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:47.516366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:47.516366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:21:47.516366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:21:47.516366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:21:47.516366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:21:47.516366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:21:47.516366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:21:47.516366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:21:47.516366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:47.516366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 399/789] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:21:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:47.516964 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d93ce20-6cb6-473a-b66e-046f85a6a130 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:47.529245 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:47.555844 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f021a-3df3-41c6-9ba9-de7821851d9f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:47.594280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b0b3c-9018-4701-9e5a-c76736e70091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:47.595357 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b0b3c-9018-4701-9e5a-c76736e70091 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:47.595932 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b0b3c-9018-4701-9e5a-c76736e70091 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:47.596405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b0b3c-9018-4701-9e5a-c76736e70091 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:47.597306 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b0b3c-9018-4701-9e5a-c76736e70091 None admin] RBAC: Adding query filter params (name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:47.597810 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b0b3c-9018-4701-9e5a-c76736e70091 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:47.599558 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b0b3c-9018-4701-9e5a-c76736e70091 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:47.608903 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 391/790] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 1 16:21:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:47.609995 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b0b3c-9018-4701-9e5a-c76736e70091 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:47.655904 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d93ce20-6cb6-473a-b66e-046f85a6a130 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:47.658385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:47.699929 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53bb76-4d86-4edc-89b0-dbb3adfedcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:47.700899 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53bb76-4d86-4edc-89b0-dbb3adfedcad None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:47.701267 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53bb76-4d86-4edc-89b0-dbb3adfedcad None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:47.701551 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53bb76-4d86-4edc-89b0-dbb3adfedcad None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:47.702401 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca53bb76-4d86-4edc-89b0-dbb3adfedcad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:47.704106 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca53bb76-4d86-4edc-89b0-dbb3adfedcad None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:47.718286 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 400/791] 10.222.0.18 () {56 vars in 1087 bytes} [Thu Sep 1 16:21:47 2022] GET /identity/v3/services => generated 772 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:47.718286 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53bb76-4d86-4edc-89b0-dbb3adfedcad None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:47.740666 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b0b3c-9018-4701-9e5a-c76736e70091 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:47.798858 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:47.834835 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53bb76-4d86-4edc-89b0-dbb3adfedcad None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:47.859006 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07a159c-5c42-4266-aada-f6acc5d12c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:47.860021 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a159c-5c42-4266-aada-f6acc5d12c65 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:47.860512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a159c-5c42-4266-aada-f6acc5d12c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:47.861007 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a159c-5c42-4266-aada-f6acc5d12c65 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:47.861915 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07a159c-5c42-4266-aada-f6acc5d12c65 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:47.862413 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07a159c-5c42-4266-aada-f6acc5d12c65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:47.864192 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07a159c-5c42-4266-aada-f6acc5d12c65 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:47.874660 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b318f9a-a411-4087-b435-0069f64e81d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:47.875355 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b318f9a-a411-4087-b435-0069f64e81d0 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:47.875649 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b318f9a-a411-4087-b435-0069f64e81d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:47.875933 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b318f9a-a411-4087-b435-0069f64e81d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:47.888324 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b318f9a-a411-4087-b435-0069f64e81d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:47.890821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b318f9a-a411-4087-b435-0069f64e81d0 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:47.907149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-2b318f9a-a411-4087-b435-0069f64e81d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:47.907149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:47.907149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:47.907149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:47.907149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:47.907149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:47.907149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:47.907149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:47.907149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:47.907149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:47.907149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:47.907149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:47.907149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:47.907149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:47.907149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:47.907149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:47.908107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:47.908107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:47.908107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:47.908107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:47.908107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:47.908107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:47.908107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:47.908107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:47.908107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:47.908107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:47.908107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:47.908107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:47.908107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:47.908107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:47.908107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:47.908107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:47.908107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:47.908985 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:47.908985 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:47.908985 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:47.908985 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 401/792] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:47.911824 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 392/793] 10.222.0.18 () {56 vars in 1135 bytes} [Thu Sep 1 16:21:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:47.915142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b318f9a-a411-4087-b435-0069f64e81d0 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:47.918075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07a159c-5c42-4266-aada-f6acc5d12c65 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:48.059842 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b318f9a-a411-4087-b435-0069f64e81d0 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:48.078598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:48.081982 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07a159c-5c42-4266-aada-f6acc5d12c65 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:48.094109 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:48.105255 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd140ce-bfff-454c-9aba-93b856ddd483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:48.106262 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd140ce-bfff-454c-9aba-93b856ddd483 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:48.106639 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd140ce-bfff-454c-9aba-93b856ddd483 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:48.106932 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd140ce-bfff-454c-9aba-93b856ddd483 None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:48.107594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd140ce-bfff-454c-9aba-93b856ddd483 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:48.107909 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd140ce-bfff-454c-9aba-93b856ddd483 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:48.109490 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd140ce-bfff-454c-9aba-93b856ddd483 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:48.123238 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 402/794] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:48.128809 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd140ce-bfff-454c-9aba-93b856ddd483 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:48.207829 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85dbfbb-68a0-4725-b608-d2460cdf4e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:48.208416 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85dbfbb-68a0-4725-b608-d2460cdf4e2c None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:48.208702 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85dbfbb-68a0-4725-b608-d2460cdf4e2c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:48.208978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85dbfbb-68a0-4725-b608-d2460cdf4e2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:48.247482 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85dbfbb-68a0-4725-b608-d2460cdf4e2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:48.247706 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd140ce-bfff-454c-9aba-93b856ddd483 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:48.247802 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85dbfbb-68a0-4725-b608-d2460cdf4e2c None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:48.280766 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f3305-97e1-4930-8946-993c97cf806b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:48.281426 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f3305-97e1-4930-8946-993c97cf806b None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:48.281713 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f3305-97e1-4930-8946-993c97cf806b None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:48.281990 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f3305-97e1-4930-8946-993c97cf806b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:48.294292 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f3305-97e1-4930-8946-993c97cf806b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:48.294292 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f3305-97e1-4930-8946-993c97cf806b None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:48.306851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-5e3f3305-97e1-4930-8946-993c97cf806b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:21:48.306851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:48.306851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:48.306851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:48.306851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:48.306851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:48.306851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:48.306851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:48.306851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:48.306851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:48.306851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:48.306851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:48.306851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:21:48.306851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:21:48.306851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:21:48.306851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:21:48.307816 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:48.307816 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:48.307816 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:21:48.307816 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:48.307816 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:21:48.307816 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:48.307816 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:48.307816 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:48.307816 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:48.307816 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:48.307816 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:48.307816 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:48.307816 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:48.307816 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:48.307816 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:48.307816 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:48.307816 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:48.307816 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:48.309128 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:48.309128 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:48.309128 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:21:48.309128 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:21:48.309128 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:21:48.309128 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:21:48.309128 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:21:48.309128 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:21:48.309128 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:21:48.309128 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:48.309128 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 403/795] 10.222.0.18 () {56 vars in 1117 bytes} [Thu Sep 1 16:21:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:21:48.313998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f3305-97e1-4930-8946-993c97cf806b None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:48.434254 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:48.521064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-a85dbfbb-68a0-4725-b608-d2460cdf4e2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:48.521064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:48.521064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:48.521064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:48.521064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:48.521064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:48.521064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:48.521064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:48.521064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:48.521064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:48.521064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:48.521064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:48.521064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:48.521064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:48.521064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:48.521064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:48.522721 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:48.522721 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:48.522721 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:48.522721 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:48.522721 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:48.522721 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:48.522721 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:48.522721 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:48.522721 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:48.522721 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:48.522721 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:48.522721 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:48.522721 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:48.522721 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:48.522721 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:48.522721 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:48.522721 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:48.523810 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:48.523810 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:48.523810 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:48.523810 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 393/796] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 604 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:21:48.529100 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85dbfbb-68a0-4725-b608-d2460cdf4e2c None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:48.583324 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f3305-97e1-4930-8946-993c97cf806b None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:48.654713 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b449bbe-d0d0-4ac0-8fd0-9169f85c63c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:48.658278 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b449bbe-d0d0-4ac0-8fd0-9169f85c63c4 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:48.658278 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b449bbe-d0d0-4ac0-8fd0-9169f85c63c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:48.658278 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b449bbe-d0d0-4ac0-8fd0-9169f85c63c4 None admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:48.658278 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b449bbe-d0d0-4ac0-8fd0-9169f85c63c4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:48.658278 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b449bbe-d0d0-4ac0-8fd0-9169f85c63c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:48.659639 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b449bbe-d0d0-4ac0-8fd0-9169f85c63c4 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:48.681245 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 404/797] 10.222.0.18 () {56 vars in 1149 bytes} [Thu Sep 1 16:21:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:21:48.683617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409e19e-e5db-487f-8d73-731f0b6caae7 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:48.683913 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409e19e-e5db-487f-8d73-731f0b6caae7 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:48.684191 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409e19e-e5db-487f-8d73-731f0b6caae7 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:48.686123 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 405/798] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:48.686955 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d409e19e-e5db-487f-8d73-731f0b6caae7 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:48.724865 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85dbfbb-68a0-4725-b608-d2460cdf4e2c None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:48.790956 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79778866-9a05-4f87-892a-9223e1468c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:48.791579 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79778866-9a05-4f87-892a-9223e1468c50 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:48.791862 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79778866-9a05-4f87-892a-9223e1468c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:48.792133 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79778866-9a05-4f87-892a-9223e1468c50 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:48.792778 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79778866-9a05-4f87-892a-9223e1468c50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:48.793142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79778866-9a05-4f87-892a-9223e1468c50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:48.794534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79778866-9a05-4f87-892a-9223e1468c50 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:48.822768 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 394/799] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:48.824128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08f32b-38ac-421b-afc6-9a027b40fc01 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:48.824422 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08f32b-38ac-421b-afc6-9a027b40fc01 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:48.824699 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08f32b-38ac-421b-afc6-9a027b40fc01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:48.866168 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-3a08f32b-38ac-421b-afc6-9a027b40fc01 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:48.867403 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3a08f32b-38ac-421b-afc6-9a027b40fc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:48.876408 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:48.895052 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d409e19e-e5db-487f-8d73-731f0b6caae7 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:48.897321 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:48.933899 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4038da51-118a-4a1d-88b6-ecb558868ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:48.934934 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038da51-118a-4a1d-88b6-ecb558868ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:48.935446 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038da51-118a-4a1d-88b6-ecb558868ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:48.935965 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038da51-118a-4a1d-88b6-ecb558868ae0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:48.941383 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 395/800] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:21:48.942225 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08f32b-38ac-421b-afc6-9a027b40fc01 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:48.948918 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4038da51-118a-4a1d-88b6-ecb558868ae0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:48.950702 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4038da51-118a-4a1d-88b6-ecb558868ae0 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:48.966482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-4038da51-118a-4a1d-88b6-ecb558868ae0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:48.966482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:48.966482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:48.966482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:48.966482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:48.966482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:48.966482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:48.966482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:48.966482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:48.966482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:48.966482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:48.966482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:48.966482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:48.966482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:48.966482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:48.966482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:48.967409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:48.967409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:48.967409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:48.967409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:48.967409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:48.967409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:48.967409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:48.967409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:48.967409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:48.967409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:48.967409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:48.967409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:48.967409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:48.967409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:48.967409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:48.967409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:48.967409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:48.968283 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:48.968283 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:48.968283 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:48.970292 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 406/801] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:48.983579 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4038da51-118a-4a1d-88b6-ecb558868ae0 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:49.052171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08f32b-38ac-421b-afc6-9a027b40fc01 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:49.091667 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e8d1b-1497-43af-89ca-400a674fd8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:49.092307 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e8d1b-1497-43af-89ca-400a674fd8a5 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:49.092592 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e8d1b-1497-43af-89ca-400a674fd8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:49.092868 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e8d1b-1497-43af-89ca-400a674fd8a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:49.110793 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e8d1b-1497-43af-89ca-400a674fd8a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:49.112226 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e8d1b-1497-43af-89ca-400a674fd8a5 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:49.132333 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-cd6e8d1b-1497-43af-89ca-400a674fd8a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:49.132333 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:49.132333 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:49.132333 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:49.132333 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:49.132333 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:49.132333 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:49.132333 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:49.132333 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:49.132333 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:49.132333 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:49.132333 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:49.132333 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:21:49.132333 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:21:49.132333 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:21:49.133259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:21:49.133259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:49.133259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:49.133259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:49.133259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:49.133259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:49.133259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:49.133259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:49.133259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:49.133259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:49.133259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:49.133259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:49.133259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:49.133259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:49.133259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:49.133259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:49.133259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:49.134113 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:21:49.134113 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:21:49.134113 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:21:49.134113 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:21:49.134113 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:21:49.134113 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:21:49.134113 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:49.134113 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:49.134113 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 396/802] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:49.139539 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e8d1b-1497-43af-89ca-400a674fd8a5 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:49.158583 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:49.179918 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4038da51-118a-4a1d-88b6-ecb558868ae0 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:49.241147 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e83847-480c-4df2-bcd1-6cf62a48bd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:49.241831 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e83847-480c-4df2-bcd1-6cf62a48bd30 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:49.242116 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e83847-480c-4df2-bcd1-6cf62a48bd30 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:49.242437 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e83847-480c-4df2-bcd1-6cf62a48bd30 None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:49.243079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e83847-480c-4df2-bcd1-6cf62a48bd30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:49.243383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e83847-480c-4df2-bcd1-6cf62a48bd30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:49.245583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e83847-480c-4df2-bcd1-6cf62a48bd30 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:49.259199 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 407/803] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:21:49.259546 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe39b9e-e3aa-4309-95c5-a03db85ef184 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:49.259880 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe39b9e-e3aa-4309-95c5-a03db85ef184 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:49.260156 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe39b9e-e3aa-4309-95c5-a03db85ef184 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:49.261568 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 408/804] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:49.266315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe39b9e-e3aa-4309-95c5-a03db85ef184 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:49.283354 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:49.295691 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e8d1b-1497-43af-89ca-400a674fd8a5 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:49.329388 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862be60e-4b6a-404d-8f8d-b8528dbabb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:49.329923 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862be60e-4b6a-404d-8f8d-b8528dbabb67 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:49.330202 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862be60e-4b6a-404d-8f8d-b8528dbabb67 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:49.330509 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862be60e-4b6a-404d-8f8d-b8528dbabb67 None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:49.331299 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862be60e-4b6a-404d-8f8d-b8528dbabb67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:49.331577 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862be60e-4b6a-404d-8f8d-b8528dbabb67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:49.333022 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862be60e-4b6a-404d-8f8d-b8528dbabb67 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:49.349562 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 397/805] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:21:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:49.350525 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862be60e-4b6a-404d-8f8d-b8528dbabb67 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:49.381984 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe39b9e-e3aa-4309-95c5-a03db85ef184 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:49.415603 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0884378c-0886-41d5-a522-01de26f98bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:49.416718 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884378c-0886-41d5-a522-01de26f98bc4 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:49.417002 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884378c-0886-41d5-a522-01de26f98bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:49.417276 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884378c-0886-41d5-a522-01de26f98bc4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:49.419576 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:49.428443 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0884378c-0886-41d5-a522-01de26f98bc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:49.433847 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-0884378c-0886-41d5-a522-01de26f98bc4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:21:49.433847 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:49.433847 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:49.433847 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:49.433847 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:49.433847 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:49.433847 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:49.433847 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:49.433847 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:49.433847 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:49.433847 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:49.433847 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:49.433847 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:49.433847 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:49.433847 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:49.433847 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:49.434998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:49.434998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:49.434998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:49.434998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:49.434998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:49.434998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:49.434998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:49.434998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:49.434998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:49.434998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:49.434998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:49.434998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:49.434998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:49.434998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:49.434998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:49.434998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:49.435839 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:49.435839 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:49.435839 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:49.435839 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:49.435839 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:49.435839 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:49.435839 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:49.435839 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:49.435839 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:49.435839 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:49.435839 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:21:49.435839 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:49.435839 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 409/806] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:21:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:49.436528 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a3ac0-75bf-4d33-8c6e-eba927bcac6e None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:49.437085 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a3ac0-75bf-4d33-8c6e-eba927bcac6e None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:49.437085 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a3ac0-75bf-4d33-8c6e-eba927bcac6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:49.454226 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-403a3ac0-75bf-4d33-8c6e-eba927bcac6e None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:49.455419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-403a3ac0-75bf-4d33-8c6e-eba927bcac6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:49.464421 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862be60e-4b6a-404d-8f8d-b8528dbabb67 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:49.501988 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48e499-a4bd-41e1-bb11-e5ba812158e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:49.502753 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48e499-a4bd-41e1-bb11-e5ba812158e4 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:49.503036 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48e499-a4bd-41e1-bb11-e5ba812158e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:49.503316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48e499-a4bd-41e1-bb11-e5ba812158e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:49.510773 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a48e499-a4bd-41e1-bb11-e5ba812158e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:49.512374 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a48e499-a4bd-41e1-bb11-e5ba812158e4 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:49.530261 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 410/807] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:21:49.530261 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a3ac0-75bf-4d33-8c6e-eba927bcac6e None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:49.530494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-4a48e499-a4bd-41e1-bb11-e5ba812158e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:49.530494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:49.530494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:49.530494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:49.530494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:49.530494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:49.530494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:49.530494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:49.530494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:49.530494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:49.530494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:49.530494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:49.530494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:21:49.530494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:21:49.530494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:21:49.531914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:21:49.531914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:49.531914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:49.531914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:49.531914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:49.531914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:49.531914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:49.531914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:49.531914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:49.531914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:49.531914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:49.531914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:49.531914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:49.531914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:49.531914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:49.531914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:49.531914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:49.533000 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:21:49.533000 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:21:49.533000 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:21:49.533000 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:21:49.533000 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:21:49.533000 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:21:49.533000 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:49.533000 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:49.533000 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 398/808] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:49.533000 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48e499-a4bd-41e1-bb11-e5ba812158e4 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:49.535195 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:49.644685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48e499-a4bd-41e1-bb11-e5ba812158e4 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:49.662485 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a3ac0-75bf-4d33-8c6e-eba927bcac6e None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:49.673327 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:49.689539 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c713d9-7d04-4f00-a9a1-3c2db3de4d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:49.690915 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c713d9-7d04-4f00-a9a1-3c2db3de4d5c None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:49.691791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c713d9-7d04-4f00-a9a1-3c2db3de4d5c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:49.692284 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c713d9-7d04-4f00-a9a1-3c2db3de4d5c None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:49.693799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c713d9-7d04-4f00-a9a1-3c2db3de4d5c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:49.694459 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c713d9-7d04-4f00-a9a1-3c2db3de4d5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:49.695854 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c713d9-7d04-4f00-a9a1-3c2db3de4d5c None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:49.705113 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9411896b-9418-4726-88d2-321ee1ce9d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:49.706117 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411896b-9418-4726-88d2-321ee1ce9d44 None admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:49.706629 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411896b-9418-4726-88d2-321ee1ce9d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:49.707097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411896b-9418-4726-88d2-321ee1ce9d44 None admin] PATH_INFO: `/v3/projects/36ce80c4e0d64562a9ab3961be76fad9/users/8137f6b2ced040bd9752239c7e987f03/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:49.709281 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 399/809] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:21:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:49.709666 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9411896b-9418-4726-88d2-321ee1ce9d44 None admin] RBAC: Authorizing `identity:create_grant(project_id=36ce80c4e0d64562a9ab3961be76fad9, user_id=8137f6b2ced040bd9752239c7e987f03, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:49.710180 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c713d9-7d04-4f00-a9a1-3c2db3de4d5c None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:49.711568 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9411896b-9418-4726-88d2-321ee1ce9d44 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:49.729207 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 411/810] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 1 16:21:49 2022] PUT /identity/v3/projects/36ce80c4e0d64562a9ab3961be76fad9/users/8137f6b2ced040bd9752239c7e987f03/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:21:49.730583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9411896b-9418-4726-88d2-321ee1ce9d44 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:49.805132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:49.857177 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9411896b-9418-4726-88d2-321ee1ce9d44 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:49.883054 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c713d9-7d04-4f00-a9a1-3c2db3de4d5c None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:49.903275 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898bf346-c3e6-41c3-9a39-f17443c99b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:49.903915 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bf346-c3e6-41c3-9a39-f17443c99b70 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:49.904131 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bf346-c3e6-41c3-9a39-f17443c99b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:49.904402 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bf346-c3e6-41c3-9a39-f17443c99b70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:49.919840 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898bf346-c3e6-41c3-9a39-f17443c99b70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:49.920596 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b81567-b984-4cb6-a5ec-24a931488dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:49.921212 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b81567-b984-4cb6-a5ec-24a931488dde None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:49.921395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898bf346-c3e6-41c3-9a39-f17443c99b70 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:49.921494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b81567-b984-4cb6-a5ec-24a931488dde None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:49.921776 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b81567-b984-4cb6-a5ec-24a931488dde None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:49.922466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b81567-b984-4cb6-a5ec-24a931488dde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:49.922773 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b81567-b984-4cb6-a5ec-24a931488dde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:49.924169 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b81567-b984-4cb6-a5ec-24a931488dde None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:49.937623 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 400/811] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:21:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:49.938675 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b81567-b984-4cb6-a5ec-24a931488dde None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:49.939787 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-898bf346-c3e6-41c3-9a39-f17443c99b70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:49.939787 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:49.939787 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:49.939787 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:49.939787 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:49.939787 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:49.939787 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:49.939787 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:49.939787 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:49.939787 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:49.939787 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:49.939787 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:49.939787 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:49.939787 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:49.939787 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:49.939787 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:49.940909 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:49.940909 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:49.940909 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:49.940909 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:49.940909 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:49.940909 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:49.940909 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:49.940909 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:49.940909 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:49.940909 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:49.940909 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:49.940909 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:49.940909 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:49.940909 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:49.940909 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:49.940909 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:49.940909 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:49.942007 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:49.942007 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:49.942007 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:49.944697 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 412/812] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:49.945799 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898bf346-c3e6-41c3-9a39-f17443c99b70 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:49.990985 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:50.056003 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b81567-b984-4cb6-a5ec-24a931488dde None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:50.073563 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898bf346-c3e6-41c3-9a39-f17443c99b70 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:50.091093 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:50.104478 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b2065-e12e-4588-af92-7b13ed25f61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:50.105256 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b2065-e12e-4588-af92-7b13ed25f61d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:50.105539 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b2065-e12e-4588-af92-7b13ed25f61d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:50.105812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b2065-e12e-4588-af92-7b13ed25f61d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:50.106680 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b2065-e12e-4588-af92-7b13ed25f61d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:50.108683 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b2065-e12e-4588-af92-7b13ed25f61d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:50.112426 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d741d7b-2e39-4dea-8e00-8ae6e1605602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:50.113259 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d741d7b-2e39-4dea-8e00-8ae6e1605602 None admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:50.113707 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d741d7b-2e39-4dea-8e00-8ae6e1605602 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:50.114168 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d741d7b-2e39-4dea-8e00-8ae6e1605602 None admin] PATH_INFO: `/v3/projects/36ce80c4e0d64562a9ab3961be76fad9/users/a5b3e53fd1314e17ad28c42a44e8eff3/roles/b6727ed962fa41109b0a2d6851551f49` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:50.119025 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-0f1b2065-e12e-4588-af92-7b13ed25f61d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:21:50.119025 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:50.119025 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:21:50.119025 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:21:50.119025 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:21:50.119025 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:21:50.119025 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:21:50.119025 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:21:50.119025 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:21:50.119025 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:50.119025 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:21:50.119025 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:50.119025 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:50.119025 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:50.119025 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:50.119025 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:50.119025 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:50.120773 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:50.120773 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:50.120773 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:50.120773 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:50.120773 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:50.120773 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:50.120773 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:21:50.120773 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:21:50.120773 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:21:50.120773 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:21:50.120773 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:50.120773 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:50.120773 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:50.120773 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:50.120773 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:50.120773 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:50.121618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:50.121618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:50.121618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:50.121618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:50.121618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:50.121618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:50.121618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:50.121618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:50.121618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:50.121618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:50.121618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:21:50.121618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:21:50.121618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:21:50.121618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:50.126339 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 401/813] 10.222.0.18 () {56 vars in 1129 bytes} [Thu Sep 1 16:21:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:50.126932 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b2065-e12e-4588-af92-7b13ed25f61d None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:50.134409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d741d7b-2e39-4dea-8e00-8ae6e1605602 None admin] RBAC: Authorizing `identity:create_grant(project_id=36ce80c4e0d64562a9ab3961be76fad9, user_id=a5b3e53fd1314e17ad28c42a44e8eff3, role_id=b6727ed962fa41109b0a2d6851551f49)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:50.135910 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d741d7b-2e39-4dea-8e00-8ae6e1605602 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:50.150728 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 413/814] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 1 16:21:49 2022] PUT /identity/v3/projects/36ce80c4e0d64562a9ab3961be76fad9/users/a5b3e53fd1314e17ad28c42a44e8eff3/roles/b6727ed962fa41109b0a2d6851551f49 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:21:50.151687 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d741d7b-2e39-4dea-8e00-8ae6e1605602 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:50.265079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b2065-e12e-4588-af92-7b13ed25f61d None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:50.278650 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:50.295209 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d741d7b-2e39-4dea-8e00-8ae6e1605602 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:50.309951 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e474e1a8-cfd9-4b46-9af1-12ddb9673600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:50.310596 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474e1a8-cfd9-4b46-9af1-12ddb9673600 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:50.310878 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474e1a8-cfd9-4b46-9af1-12ddb9673600 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:50.311147 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474e1a8-cfd9-4b46-9af1-12ddb9673600 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:50.312561 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e474e1a8-cfd9-4b46-9af1-12ddb9673600 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:50.312684 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e474e1a8-cfd9-4b46-9af1-12ddb9673600 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:50.314157 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e474e1a8-cfd9-4b46-9af1-12ddb9673600 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:50.328369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 402/815] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:50.331122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce4f27-415c-4d6e-b8d1-a37c95d58305 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:50.331556 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce4f27-415c-4d6e-b8d1-a37c95d58305 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:50.331997 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce4f27-415c-4d6e-b8d1-a37c95d58305 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:50.333703 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 403/816] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:50.334614 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce4f27-415c-4d6e-b8d1-a37c95d58305 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:50.349706 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a76bb5-94bd-4f4a-972d-c6a8e7ad59e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:50.349706 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a76bb5-94bd-4f4a-972d-c6a8e7ad59e8 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:50.349706 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a76bb5-94bd-4f4a-972d-c6a8e7ad59e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:50.349706 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a76bb5-94bd-4f4a-972d-c6a8e7ad59e8 None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:50.350097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a76bb5-94bd-4f4a-972d-c6a8e7ad59e8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:50.351042 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a76bb5-94bd-4f4a-972d-c6a8e7ad59e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:50.352457 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a76bb5-94bd-4f4a-972d-c6a8e7ad59e8 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:50.360481 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 414/817] 10.222.0.18 () {56 vars in 1124 bytes} [Thu Sep 1 16:21:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:50.360481 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226fe23-3d1a-4615-847a-964b4ad21d2f None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:50.360481 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226fe23-3d1a-4615-847a-964b4ad21d2f None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:50.360481 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226fe23-3d1a-4615-847a-964b4ad21d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:50.384602 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-4226fe23-3d1a-4615-847a-964b4ad21d2f None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:50.385508 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-4226fe23-3d1a-4615-847a-964b4ad21d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:50.409713 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:50.462262 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 415/818] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:21:50.462262 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4226fe23-3d1a-4615-847a-964b4ad21d2f None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:50.464009 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce4f27-415c-4d6e-b8d1-a37c95d58305 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:50.527068 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ebfa8-19c4-41c1-8e93-adcfdfce9600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:50.527619 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ebfa8-19c4-41c1-8e93-adcfdfce9600 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:50.528104 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ebfa8-19c4-41c1-8e93-adcfdfce9600 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:50.528381 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ebfa8-19c4-41c1-8e93-adcfdfce9600 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:50.543210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ebfa8-19c4-41c1-8e93-adcfdfce9600 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:50.544661 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ebfa8-19c4-41c1-8e93-adcfdfce9600 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:50.566210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-2c2ebfa8-19c4-41c1-8e93-adcfdfce9600 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:21:50.566210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:50.566210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:50.566210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:50.566210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:50.566210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:50.566210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:50.566210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:50.566210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:50.566210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:50.566210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:50.566210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:50.566210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:21:50.566210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:21:50.566210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:21:50.566210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:21:50.567144 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:50.567144 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:50.567144 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:21:50.567144 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:50.567144 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:21:50.567144 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:50.567144 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:50.567144 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:50.567144 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:50.567144 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:50.567144 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:50.567144 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:50.567144 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:50.567144 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:50.567144 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:50.567144 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:50.567144 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:50.567144 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:50.568056 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:50.568056 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:50.568056 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:21:50.568056 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:21:50.568056 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:21:50.568056 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:21:50.568056 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:21:50.568056 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:21:50.568056 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:21:50.568056 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:50.568056 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 404/819] 10.222.0.18 () {56 vars in 1093 bytes} [Thu Sep 1 16:21:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:50.574255 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ebfa8-19c4-41c1-8e93-adcfdfce9600 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:50.606406 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4226fe23-3d1a-4615-847a-964b4ad21d2f None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:50.625047 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:50.678963 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d499281-0fc3-4757-9325-b53650bea049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:50.679587 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d499281-0fc3-4757-9325-b53650bea049 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:50.679873 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d499281-0fc3-4757-9325-b53650bea049 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:50.680152 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d499281-0fc3-4757-9325-b53650bea049 None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:50.680916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d499281-0fc3-4757-9325-b53650bea049 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:50.681224 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d499281-0fc3-4757-9325-b53650bea049 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:50.682779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d499281-0fc3-4757-9325-b53650bea049 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:50.694422 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 416/820] 10.222.0.18 () {56 vars in 1124 bytes} [Thu Sep 1 16:21:50 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:50.699190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d499281-0fc3-4757-9325-b53650bea049 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:50.723905 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ebfa8-19c4-41c1-8e93-adcfdfce9600 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:50.747041 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:50.795047 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be5065-1e8d-476d-81dc-14d586157114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:50.795702 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be5065-1e8d-476d-81dc-14d586157114 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:50.795982 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be5065-1e8d-476d-81dc-14d586157114 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:50.796250 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be5065-1e8d-476d-81dc-14d586157114 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:50.796976 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76be5065-1e8d-476d-81dc-14d586157114 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:50.797281 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76be5065-1e8d-476d-81dc-14d586157114 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:50.798734 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76be5065-1e8d-476d-81dc-14d586157114 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:50.828360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 405/821] 10.222.0.18 () {56 vars in 1133 bytes} [Thu Sep 1 16:21:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:50.829347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be5065-1e8d-476d-81dc-14d586157114 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:50.865232 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d499281-0fc3-4757-9325-b53650bea049 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:50.910728 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b086e-265e-490b-b678-d7bef65a1e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:50.912880 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b086e-265e-490b-b678-d7bef65a1e72 None admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:50.913380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b086e-265e-490b-b678-d7bef65a1e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:50.913872 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b086e-265e-490b-b678-d7bef65a1e72 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:50.915011 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b086e-265e-490b-b678-d7bef65a1e72 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:50.917314 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b086e-265e-490b-b678-d7bef65a1e72 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:50.939215 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:50.947629 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 417/822] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:21:50 2022] POST /identity/v3/endpoints => generated 346 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:21:50.950128 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b086e-265e-490b-b678-d7bef65a1e72 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:50.959513 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be5065-1e8d-476d-81dc-14d586157114 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:51.006323 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d34e6-adff-48fd-85ed-2ca94adac36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:51.006918 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d34e6-adff-48fd-85ed-2ca94adac36c None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:51.007194 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d34e6-adff-48fd-85ed-2ca94adac36c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:51.007459 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d34e6-adff-48fd-85ed-2ca94adac36c None admin] PATH_INFO: `/v3/services/network` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:51.008044 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d34e6-adff-48fd-85ed-2ca94adac36c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:51.009419 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d34e6-adff-48fd-85ed-2ca94adac36c None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:51.020891 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-8d4d34e6-adff-48fd-85ed-2ca94adac36c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:21:51.020891 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:51.020891 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:21:51.020891 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:21:51.020891 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:21:51.020891 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:21:51.020891 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:21:51.020891 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:21:51.020891 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:21:51.020891 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:51.020891 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:21:51.020891 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:51.020891 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:51.020891 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:51.020891 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:51.020891 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:51.020891 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:51.021896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:51.021896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:51.021896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:51.021896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:51.021896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:51.021896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:51.021896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:21:51.021896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:21:51.021896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:21:51.021896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:21:51.021896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:51.021896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:51.021896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:51.021896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:51.021896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:51.021896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:51.022735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:51.022735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:51.022735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:51.022735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:51.022735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:51.022735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:51.022735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:51.022735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:51.022735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:51.022735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:51.022735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:21:51.022735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:21:51.022735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:21:51.022735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:51.026451 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 406/823] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:50 2022] GET /identity/v3/services/network => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:51.028597 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d34e6-adff-48fd-85ed-2ca94adac36c None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:51.067052 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:51.091082 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b086e-265e-490b-b678-d7bef65a1e72 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:51.127423 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974fa0b-5744-4cad-b7f1-beb0b54dc693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:51.128111 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974fa0b-5744-4cad-b7f1-beb0b54dc693 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:51.128395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974fa0b-5744-4cad-b7f1-beb0b54dc693 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:51.128668 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974fa0b-5744-4cad-b7f1-beb0b54dc693 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:51.164362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d34e6-adff-48fd-85ed-2ca94adac36c None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:51.207662 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c974fa0b-5744-4cad-b7f1-beb0b54dc693 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:51.207864 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf5dce-003e-4a18-8ad9-174fd83b4b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:51.208542 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf5dce-003e-4a18-8ad9-174fd83b4b6e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:51.208825 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf5dce-003e-4a18-8ad9-174fd83b4b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:51.209141 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf5dce-003e-4a18-8ad9-174fd83b4b6e None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:51.209770 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbf5dce-003e-4a18-8ad9-174fd83b4b6e None admin] RBAC: Adding query filter params (name=network) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:51.210071 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbf5dce-003e-4a18-8ad9-174fd83b4b6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:51.210764 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c974fa0b-5744-4cad-b7f1-beb0b54dc693 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:51.211437 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbf5dce-003e-4a18-8ad9-174fd83b4b6e None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:51.222432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 407/824] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Sep 1 16:21:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:21:51.223998 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf5dce-003e-4a18-8ad9-174fd83b4b6e None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:51.240733 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:51.304560 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:51.328902 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 418/825] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:21:50 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:21:51.334470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974fa0b-5744-4cad-b7f1-beb0b54dc693 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:51.368059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf5dce-003e-4a18-8ad9-174fd83b4b6e None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:51.412061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69c649-adaf-4161-910a-611c08feed7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:51.413023 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69c649-adaf-4161-910a-611c08feed7e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:51.413470 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69c649-adaf-4161-910a-611c08feed7e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:51.413929 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69c649-adaf-4161-910a-611c08feed7e None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:51.414872 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a69c649-adaf-4161-910a-611c08feed7e None admin] RBAC: Adding query filter params (type=network) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:51.415350 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a69c649-adaf-4161-910a-611c08feed7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:51.416903 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a69c649-adaf-4161-910a-611c08feed7e None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:51.426335 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 408/826] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Sep 1 16:21:51 2022] GET /identity/v3/services?type=network => generated 350 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:51.428235 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69c649-adaf-4161-910a-611c08feed7e None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:51.457788 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974fa0b-5744-4cad-b7f1-beb0b54dc693 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:51.498052 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d143431-5add-4e59-bbda-d28ef7191737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:51.499124 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d143431-5add-4e59-bbda-d28ef7191737 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:51.499124 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d143431-5add-4e59-bbda-d28ef7191737 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:51.499337 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d143431-5add-4e59-bbda-d28ef7191737 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:51.510921 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d143431-5add-4e59-bbda-d28ef7191737 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:51.512301 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:51.513413 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d143431-5add-4e59-bbda-d28ef7191737 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:51.532108 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-5d143431-5add-4e59-bbda-d28ef7191737 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:51.532108 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:51.532108 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:51.532108 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:51.532108 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:51.532108 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:51.532108 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:51.532108 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:51.532108 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:51.532108 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:51.532108 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:51.532108 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:51.532108 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:51.532108 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:51.532108 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:51.532108 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:51.533285 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:51.533285 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:51.533285 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:51.533285 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:51.533285 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:51.533285 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:51.533285 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:51.533285 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:51.533285 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:51.533285 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:51.533285 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:51.533285 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:51.533285 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:51.533285 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:51.533285 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:51.533285 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:51.533285 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:51.534599 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:51.534599 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:51.534599 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:51.534599 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 419/827] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:21:51.539025 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d143431-5add-4e59-bbda-d28ef7191737 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:51.545264 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69c649-adaf-4161-910a-611c08feed7e None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:51.580755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:51.582365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c6b94-3107-4db4-aee4-ad9216ef9ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:51.583467 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c6b94-3107-4db4-aee4-ad9216ef9ced None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:51.583940 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c6b94-3107-4db4-aee4-ad9216ef9ced None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:51.584372 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c6b94-3107-4db4-aee4-ad9216ef9ced None admin] PATH_INFO: `/v3/endpoints` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:51.585198 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c6b94-3107-4db4-aee4-ad9216ef9ced None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:51.587282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c6b94-3107-4db4-aee4-ad9216ef9ced None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:51.618108 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 409/828] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:21:51 2022] POST /identity/v3/endpoints => generated 337 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:21:51.663738 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d143431-5add-4e59-bbda-d28ef7191737 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:51.707253 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528dcf8-156f-489d-9bbc-a0262a493d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:51.707890 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528dcf8-156f-489d-9bbc-a0262a493d1a None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:51.708162 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528dcf8-156f-489d-9bbc-a0262a493d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:51.708435 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528dcf8-156f-489d-9bbc-a0262a493d1a None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:51.709146 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528dcf8-156f-489d-9bbc-a0262a493d1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:51.709458 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528dcf8-156f-489d-9bbc-a0262a493d1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:51.710869 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528dcf8-156f-489d-9bbc-a0262a493d1a None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:51.724885 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 420/829] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:51.731436 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c6b94-3107-4db4-aee4-ad9216ef9ced None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:51.863113 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c6b94-3107-4db4-aee4-ad9216ef9ced None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:51.896015 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:51.902135 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d0d3b7-1949-4443-a43d-f8643845cbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:51.902904 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0d3b7-1949-4443-a43d-f8643845cbd4 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:51.903180 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0d3b7-1949-4443-a43d-f8643845cbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:51.903449 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0d3b7-1949-4443-a43d-f8643845cbd4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:51.910677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d0d3b7-1949-4443-a43d-f8643845cbd4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:51.912124 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d0d3b7-1949-4443-a43d-f8643845cbd4 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:51.926132 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-08d0d3b7-1949-4443-a43d-f8643845cbd4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:51.926132 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:51.926132 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:51.926132 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:51.926132 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:51.926132 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:51.926132 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:51.926132 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:51.926132 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:51.926132 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:51.926132 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:51.926132 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:51.926132 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:21:51.926132 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:21:51.926132 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:21:51.927251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:21:51.927251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:51.927251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:51.927251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:51.927251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:51.927251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:51.927251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:51.927251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:51.927251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:51.927251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:51.927251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:51.927251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:51.927251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:51.927251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:51.927251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:51.927251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:51.927251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:51.928320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:21:51.928320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:21:51.928320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:21:51.928320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:21:51.928320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:21:51.928320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:21:51.928320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:51.928320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:51.930720 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 410/830] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:51.936572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528dcf8-156f-489d-9bbc-a0262a493d1a None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:52.007554 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6fe0c-4ab4-4509-86ff-484e603281e9 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:52.008182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6fe0c-4ab4-4509-86ff-484e603281e9 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:52.008631 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6fe0c-4ab4-4509-86ff-484e603281e9 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:52.010756 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 411/831] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:21:52.016077 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11afde7c-6285-4985-b5cc-9537fbef0066 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:52.016628 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11afde7c-6285-4985-b5cc-9537fbef0066 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:52.017077 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11afde7c-6285-4985-b5cc-9537fbef0066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:52.038973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-11afde7c-6285-4985-b5cc-9537fbef0066 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:52.040443 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-11afde7c-6285-4985-b5cc-9537fbef0066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:52.047284 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:52.062040 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528dcf8-156f-489d-9bbc-a0262a493d1a None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:52.099238 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2708e8a-c6ee-41fc-b8d8-3986a10ead63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:52.099854 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2708e8a-c6ee-41fc-b8d8-3986a10ead63 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:52.100134 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2708e8a-c6ee-41fc-b8d8-3986a10ead63 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:52.100404 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2708e8a-c6ee-41fc-b8d8-3986a10ead63 None admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:52.101199 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2708e8a-c6ee-41fc-b8d8-3986a10ead63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:52.101509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2708e8a-c6ee-41fc-b8d8-3986a10ead63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:52.103073 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2708e8a-c6ee-41fc-b8d8-3986a10ead63 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:52.116339 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 421/832] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:21:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:52.125933 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2708e8a-c6ee-41fc-b8d8-3986a10ead63 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:52.141567 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 412/833] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:21:52.226648 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2708e8a-c6ee-41fc-b8d8-3986a10ead63 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:52.264587 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495cc6a2-8f08-4baa-916f-bd388f20ff12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:52.265197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495cc6a2-8f08-4baa-916f-bd388f20ff12 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:52.265476 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495cc6a2-8f08-4baa-916f-bd388f20ff12 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:52.265749 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495cc6a2-8f08-4baa-916f-bd388f20ff12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:52.266507 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495cc6a2-8f08-4baa-916f-bd388f20ff12 None admin] RBAC: Adding query filter params (role.id=e8ba063905484d22993bcf04830091a3, scope.project.id=36ce80c4e0d64562a9ab3961be76fad9, user.id=278e6d6aed414c758fe3e9f4df6bf81e) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:52.266821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495cc6a2-8f08-4baa-916f-bd388f20ff12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:52.268258 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495cc6a2-8f08-4baa-916f-bd388f20ff12 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:52.274415 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 422/834] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:21:52 2022] GET /identity/v3/role_assignments?user.id=278e6d6aed414c758fe3e9f4df6bf81e&scope.project.id=36ce80c4e0d64562a9ab3961be76fad9&role.id=e8ba063905484d22993bcf04830091a3 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:52.328487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09dc8c-9f10-478f-877a-b097efcb3065 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:52.328842 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09dc8c-9f10-478f-877a-b097efcb3065 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:52.329056 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09dc8c-9f10-478f-877a-b097efcb3065 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:52.330711 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 413/835] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:52.336312 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3af777-d90b-4808-bc17-47f446bd4d90 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:52.336631 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3af777-d90b-4808-bc17-47f446bd4d90 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:52.336887 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3af777-d90b-4808-bc17-47f446bd4d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:52.357153 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-ab3af777-d90b-4808-bc17-47f446bd4d90 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:52.358417 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-ab3af777-d90b-4808-bc17-47f446bd4d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:52.367258 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:52.441184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09dc8c-9f10-478f-877a-b097efcb3065 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:52.442282 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 423/836] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:21:52.553665 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09dc8c-9f10-478f-877a-b097efcb3065 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:52.575010 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:52.591244 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9409cd-c2b5-4e32-99ef-38190d5d87de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:52.591900 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9409cd-c2b5-4e32-99ef-38190d5d87de None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:52.592179 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9409cd-c2b5-4e32-99ef-38190d5d87de None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:52.592449 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9409cd-c2b5-4e32-99ef-38190d5d87de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:52.605346 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9409cd-c2b5-4e32-99ef-38190d5d87de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:52.610262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-9f9409cd-c2b5-4e32-99ef-38190d5d87de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:21:52.610262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:52.610262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:52.610262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:52.610262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:52.610262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:52.610262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:52.610262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:52.610262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:52.610262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:52.610262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:52.610262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:52.610262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:52.610262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:52.610262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:52.610262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:52.611347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:52.611347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:52.611347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:52.611347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:52.611347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:52.611347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:52.611347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:52.611347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:52.611347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:52.611347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:52.611347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:52.611347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:52.611347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:52.611347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:52.611347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:52.611347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:52.612283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:52.612283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:52.612283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:52.612283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:52.612283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:52.612283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:52.612283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:52.612283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:52.612283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:52.612283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:52.612283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:21:52.612283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:52.612283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 414/837] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:21:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:21:52.618834 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3af777-d90b-4808-bc17-47f446bd4d90 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:52.714748 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9409cd-c2b5-4e32-99ef-38190d5d87de None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:52.730394 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3af777-d90b-4808-bc17-47f446bd4d90 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:52.771164 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb589e-c31c-4fad-a6db-b4d81ab44cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:52.771776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb589e-c31c-4fad-a6db-b4d81ab44cff None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:52.772054 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb589e-c31c-4fad-a6db-b4d81ab44cff None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:52.772327 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb589e-c31c-4fad-a6db-b4d81ab44cff None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:52.773511 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb589e-c31c-4fad-a6db-b4d81ab44cff None admin] RBAC: Adding query filter params (name=admin) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:52.773823 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb589e-c31c-4fad-a6db-b4d81ab44cff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:52.776036 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb589e-c31c-4fad-a6db-b4d81ab44cff None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:52.788666 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 424/838] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:21:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:52.793821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb589e-c31c-4fad-a6db-b4d81ab44cff None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:52.798530 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:52.828300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9409cd-c2b5-4e32-99ef-38190d5d87de None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:52.872788 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ebc5be-0839-4c9b-bbac-103d84b116a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:52.873429 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebc5be-0839-4c9b-bbac-103d84b116a0 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:52.873711 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebc5be-0839-4c9b-bbac-103d84b116a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:52.873985 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebc5be-0839-4c9b-bbac-103d84b116a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:52.888072 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ebc5be-0839-4c9b-bbac-103d84b116a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:52.895562 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-95ebc5be-0839-4c9b-bbac-103d84b116a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:52.895562 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:52.895562 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:52.895562 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:52.895562 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:52.895562 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:52.895562 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:52.895562 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:52.895562 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:52.895562 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:52.895562 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:52.895562 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:52.895562 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:52.895562 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:52.895562 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:52.895562 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:52.896475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:52.896475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:52.896475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:52.896475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:52.896475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:52.896475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:52.896475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:52.896475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:52.896475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:52.896475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:52.896475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:52.896475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:52.896475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:52.896475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:52.896475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:52.896475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:52.903079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:52.903079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:52.903079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:52.903079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:52.903079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:52.903079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:52.903079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:52.903079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:52.903079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:52.903079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:52.903079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:21:52.903079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:52.903079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 415/839] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 1 16:21:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:52.903758 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ebc5be-0839-4c9b-bbac-103d84b116a0 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:52.918170 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb589e-c31c-4fad-a6db-b4d81ab44cff None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:52.951599 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:52.968555 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012f66e-84ed-42c1-9f6e-4966583e2476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:52.969189 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012f66e-84ed-42c1-9f6e-4966583e2476 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:52.969471 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012f66e-84ed-42c1-9f6e-4966583e2476 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:52.969876 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012f66e-84ed-42c1-9f6e-4966583e2476 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:52.985121 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012f66e-84ed-42c1-9f6e-4966583e2476 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:52.986617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012f66e-84ed-42c1-9f6e-4966583e2476 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:53.005588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-8012f66e-84ed-42c1-9f6e-4966583e2476 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:53.005588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:53.005588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:53.005588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:53.005588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:53.005588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:53.005588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:53.005588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:53.005588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:53.005588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:53.005588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:53.005588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:53.005588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:53.005588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:53.005588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:53.005588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:53.006701 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:53.006701 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:53.006701 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:53.006701 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:53.006701 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:53.006701 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:53.006701 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:53.006701 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:53.006701 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:53.006701 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:53.006701 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:53.006701 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:53.006701 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:53.006701 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:53.006701 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:53.006701 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:53.006701 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:53.007600 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:53.007600 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:53.007600 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:53.010470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 425/840] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:53.013452 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012f66e-84ed-42c1-9f6e-4966583e2476 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:53.023363 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ebc5be-0839-4c9b-bbac-103d84b116a0 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:53.069261 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944be92a-ed34-4de7-acae-6c55521440f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:53.070479 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944be92a-ed34-4de7-acae-6c55521440f1 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:53.071353 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944be92a-ed34-4de7-acae-6c55521440f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:53.072481 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944be92a-ed34-4de7-acae-6c55521440f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:53.073414 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944be92a-ed34-4de7-acae-6c55521440f1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:53.073920 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944be92a-ed34-4de7-acae-6c55521440f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:53.076109 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944be92a-ed34-4de7-acae-6c55521440f1 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:53.081927 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:53.089705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 416/841] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 1 16:21:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:21:53.094877 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944be92a-ed34-4de7-acae-6c55521440f1 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:53.140293 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012f66e-84ed-42c1-9f6e-4966583e2476 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:53.178517 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203f52b-dfc5-49c5-ac8b-1d013280f188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:53.179160 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203f52b-dfc5-49c5-ac8b-1d013280f188 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:53.179857 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203f52b-dfc5-49c5-ac8b-1d013280f188 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:53.179857 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203f52b-dfc5-49c5-ac8b-1d013280f188 None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:53.180344 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203f52b-dfc5-49c5-ac8b-1d013280f188 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:53.180650 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203f52b-dfc5-49c5-ac8b-1d013280f188 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:53.182085 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203f52b-dfc5-49c5-ac8b-1d013280f188 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:53.200443 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 426/842] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:53.201449 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203f52b-dfc5-49c5-ac8b-1d013280f188 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:53.224278 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944be92a-ed34-4de7-acae-6c55521440f1 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:53.225954 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:53.262286 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ef248-c646-4331-8bd8-4ad5cd5ae01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:53.262977 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ef248-c646-4331-8bd8-4ad5cd5ae01d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:53.263518 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ef248-c646-4331-8bd8-4ad5cd5ae01d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:53.263518 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ef248-c646-4331-8bd8-4ad5cd5ae01d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:53.274839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722ef248-c646-4331-8bd8-4ad5cd5ae01d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:53.276380 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722ef248-c646-4331-8bd8-4ad5cd5ae01d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:53.291170 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-722ef248-c646-4331-8bd8-4ad5cd5ae01d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:53.291170 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:53.291170 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:53.291170 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:53.291170 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:53.291170 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:53.291170 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:53.291170 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:53.291170 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:53.291170 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:53.291170 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:53.291170 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:53.291170 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:53.291170 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:53.291170 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:53.291170 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:53.292119 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:53.292119 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:53.292119 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:53.292119 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:53.292119 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:53.292119 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:53.292119 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:53.292119 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:53.292119 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:53.292119 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:53.292119 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:53.292119 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:53.292119 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:53.292119 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:53.292119 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:53.292119 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:53.292119 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:53.292969 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:53.292969 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:53.292969 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:53.297472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 417/843] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:21:53.298398 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ef248-c646-4331-8bd8-4ad5cd5ae01d None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:53.323031 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203f52b-dfc5-49c5-ac8b-1d013280f188 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:53.340537 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:53.360528 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f645f-9767-435d-a718-474bb5579c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:53.361525 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f645f-9767-435d-a718-474bb5579c88 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:53.362031 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f645f-9767-435d-a718-474bb5579c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:53.362540 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f645f-9767-435d-a718-474bb5579c88 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:53.371392 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f645f-9767-435d-a718-474bb5579c88 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:53.373071 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f645f-9767-435d-a718-474bb5579c88 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:53.386003 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-5e5f645f-9767-435d-a718-474bb5579c88 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:21:53.386003 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:53.386003 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:53.386003 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:53.386003 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:53.386003 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:53.386003 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:53.386003 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:53.386003 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:53.386003 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:53.386003 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:53.386003 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:53.386003 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:21:53.386003 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:21:53.386003 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:21:53.386003 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:21:53.386956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:53.386956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:53.386956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:21:53.386956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:53.386956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:21:53.386956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:53.386956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:53.386956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:53.386956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:53.386956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:53.386956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:53.386956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:53.386956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:53.386956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:53.386956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:53.386956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:53.386956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:53.386956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:53.387849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:53.387849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:53.387849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:21:53.387849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:21:53.387849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:21:53.387849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:21:53.387849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:21:53.387849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:21:53.387849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:21:53.387849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:53.394744 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 427/844] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:21:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:53.395801 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f645f-9767-435d-a718-474bb5579c88 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:53.409444 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ef248-c646-4331-8bd8-4ad5cd5ae01d None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:53.445704 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168b177-6d00-4709-b10f-eb4c2e256795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:53.446347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168b177-6d00-4709-b10f-eb4c2e256795 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:53.446634 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168b177-6d00-4709-b10f-eb4c2e256795 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:53.447465 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168b177-6d00-4709-b10f-eb4c2e256795 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:53.448123 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9168b177-6d00-4709-b10f-eb4c2e256795 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:53.448484 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9168b177-6d00-4709-b10f-eb4c2e256795 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:53.449837 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9168b177-6d00-4709-b10f-eb4c2e256795 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:53.466381 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 418/845] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:53.467657 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bc6a0-3c8a-4d31-9284-e6c06c7bbb79 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:53.467918 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bc6a0-3c8a-4d31-9284-e6c06c7bbb79 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:53.468187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bc6a0-3c8a-4d31-9284-e6c06c7bbb79 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:53.469396 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 419/846] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:53.470073 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606bc6a0-3c8a-4d31-9284-e6c06c7bbb79 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:53.486012 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:53.511220 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f645f-9767-435d-a718-474bb5579c88 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:53.548261 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48932656-d9fb-4ce1-9b56-1c632a5b94fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:53.548874 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48932656-d9fb-4ce1-9b56-1c632a5b94fd None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:53.549153 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48932656-d9fb-4ce1-9b56-1c632a5b94fd None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:53.549481 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48932656-d9fb-4ce1-9b56-1c632a5b94fd None admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:53.550212 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48932656-d9fb-4ce1-9b56-1c632a5b94fd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:53.550548 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48932656-d9fb-4ce1-9b56-1c632a5b94fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:53.551983 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48932656-d9fb-4ce1-9b56-1c632a5b94fd None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:53.574712 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 428/847] 10.222.0.18 () {56 vars in 1135 bytes} [Thu Sep 1 16:21:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:53.574712 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e567ce-dfeb-4137-b918-28c9eed2ac52 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:53.574712 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e567ce-dfeb-4137-b918-28c9eed2ac52 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:53.574712 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e567ce-dfeb-4137-b918-28c9eed2ac52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:53.584672 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606bc6a0-3c8a-4d31-9284-e6c06c7bbb79 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:53.590797 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-f2e567ce-dfeb-4137-b918-28c9eed2ac52 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:53.592220 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-f2e567ce-dfeb-4137-b918-28c9eed2ac52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:53.619434 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef05dd9-5fc0-465a-ae59-ac4a3b399deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:53.620318 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef05dd9-5fc0-465a-ae59-ac4a3b399deb None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:53.620594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef05dd9-5fc0-465a-ae59-ac4a3b399deb None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:53.620865 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef05dd9-5fc0-465a-ae59-ac4a3b399deb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:53.621748 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:53.628513 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef05dd9-5fc0-465a-ae59-ac4a3b399deb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:53.629910 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef05dd9-5fc0-465a-ae59-ac4a3b399deb None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:53.642665 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-4ef05dd9-5fc0-465a-ae59-ac4a3b399deb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:21:53.642665 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:53.642665 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:53.642665 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:53.642665 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:53.642665 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:53.642665 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:53.642665 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:53.642665 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:53.642665 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:53.642665 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:53.642665 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:53.642665 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:21:53.642665 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:21:53.642665 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:21:53.642665 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:21:53.643566 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:53.643566 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:53.643566 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:21:53.643566 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:53.643566 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:21:53.643566 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:53.643566 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:53.643566 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:53.643566 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:53.643566 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:53.643566 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:53.643566 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:53.643566 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:53.643566 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:53.643566 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:53.643566 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:53.643566 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:53.643566 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:53.644475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:53.644475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:53.644475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:21:53.644475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:21:53.644475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:21:53.644475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:21:53.644475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:21:53.644475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:21:53.644475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:21:53.644475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:53.644475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 420/848] 10.222.0.18 () {56 vars in 1117 bytes} [Thu Sep 1 16:21:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:53.644475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef05dd9-5fc0-465a-ae59-ac4a3b399deb None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:53.671080 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 429/849] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:21:53.671688 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e567ce-dfeb-4137-b918-28c9eed2ac52 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:53.748836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:53.782595 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef05dd9-5fc0-465a-ae59-ac4a3b399deb None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:53.791430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e567ce-dfeb-4137-b918-28c9eed2ac52 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:53.825818 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879ced9-0dda-43d3-929d-c87cdd043966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:53.826455 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879ced9-0dda-43d3-929d-c87cdd043966 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:53.826760 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879ced9-0dda-43d3-929d-c87cdd043966 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:53.827081 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879ced9-0dda-43d3-929d-c87cdd043966 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:53.837194 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35917dd6-7598-4eda-9c90-e15a8da54c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:53.838523 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35917dd6-7598-4eda-9c90-e15a8da54c4f None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:53.839013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35917dd6-7598-4eda-9c90-e15a8da54c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:53.839870 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35917dd6-7598-4eda-9c90-e15a8da54c4f None admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:53.841062 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35917dd6-7598-4eda-9c90-e15a8da54c4f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:53.841555 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35917dd6-7598-4eda-9c90-e15a8da54c4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:53.844000 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35917dd6-7598-4eda-9c90-e15a8da54c4f None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:53.868258 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 430/850] 10.222.0.18 () {56 vars in 1149 bytes} [Thu Sep 1 16:21:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:53.875391 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35917dd6-7598-4eda-9c90-e15a8da54c4f None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:53.900210 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:53.912385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879ced9-0dda-43d3-929d-c87cdd043966 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:53.915025 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879ced9-0dda-43d3-929d-c87cdd043966 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:53.983235 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35917dd6-7598-4eda-9c90-e15a8da54c4f None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:53.999109 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:54.021438 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432833de-172e-45e3-9b1a-affbb4696fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:54.022057 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432833de-172e-45e3-9b1a-affbb4696fce None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:54.022363 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432833de-172e-45e3-9b1a-affbb4696fce None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:54.022640 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432833de-172e-45e3-9b1a-affbb4696fce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:54.027221 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 421/851] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:21:53 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:21:54.027771 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879ced9-0dda-43d3-929d-c87cdd043966 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:54.099535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432833de-172e-45e3-9b1a-affbb4696fce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:54.101133 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432833de-172e-45e3-9b1a-affbb4696fce None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:54.126441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:54.130724 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879ced9-0dda-43d3-929d-c87cdd043966 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:54.166269 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b62eb-2273-4931-ba94-b0ff421e0c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:54.166905 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b62eb-2273-4931-ba94-b0ff421e0c41 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:54.167195 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b62eb-2273-4931-ba94-b0ff421e0c41 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:54.167474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b62eb-2273-4931-ba94-b0ff421e0c41 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:54.168056 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b62eb-2273-4931-ba94-b0ff421e0c41 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:54.169408 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b62eb-2273-4931-ba94-b0ff421e0c41 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:54.179039 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-789b62eb-2273-4931-ba94-b0ff421e0c41 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:21:54.179039 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:54.179039 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:21:54.179039 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:21:54.179039 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:21:54.179039 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:21:54.179039 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:21:54.179039 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:21:54.179039 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:21:54.179039 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:54.179039 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:21:54.179039 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:54.179039 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:54.179039 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:54.179039 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:54.179039 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:54.179039 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:54.180029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:54.180029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:54.180029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:54.180029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:54.180029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:54.180029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:54.180029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:21:54.180029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:21:54.180029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:21:54.180029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:21:54.180029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:54.180029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:54.180029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:54.180029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:54.180029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:54.180029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:54.180836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:54.180836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:54.180836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:54.180836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:54.180836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:54.180836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:54.180836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:54.180836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:54.180836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:54.180836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:54.180836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:21:54.180836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:21:54.180836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:21:54.180836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:54.180836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 422/852] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 1 16:21:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:54.181678 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b62eb-2273-4931-ba94-b0ff421e0c41 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:54.218409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 431/853] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:21:53 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:21:54.219280 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432833de-172e-45e3-9b1a-affbb4696fce None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:54.230160 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:54.295786 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b62eb-2273-4931-ba94-b0ff421e0c41 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:54.323925 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432833de-172e-45e3-9b1a-affbb4696fce None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:54.332418 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa950363-7eb3-4419-ba03-e4d140305c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:54.333051 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa950363-7eb3-4419-ba03-e4d140305c65 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:54.333343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa950363-7eb3-4419-ba03-e4d140305c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:54.333625 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa950363-7eb3-4419-ba03-e4d140305c65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:54.344481 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa950363-7eb3-4419-ba03-e4d140305c65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:54.345606 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa950363-7eb3-4419-ba03-e4d140305c65 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:54.347018 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:54.359463 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320ca0e-c5d1-4851-a90b-5b4b2078f3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:54.360079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320ca0e-c5d1-4851-a90b-5b4b2078f3cd None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:54.360195 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-fa950363-7eb3-4419-ba03-e4d140305c65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:54.360195 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:54.360195 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:54.360195 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:54.360195 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:54.360195 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:54.360195 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:54.360195 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:54.360195 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:54.360195 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:54.360195 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:54.360195 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:54.360195 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:54.360195 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:54.360195 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:54.360195 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:54.361141 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:54.361141 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:54.361141 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:54.361141 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:54.361141 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:54.361141 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:54.361141 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:54.361141 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:54.361141 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:54.361141 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:54.361141 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:54.361141 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:54.361141 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:54.361141 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:54.361141 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:54.361141 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:54.361141 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:54.362012 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:54.362012 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:54.362012 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:54.362176 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320ca0e-c5d1-4851-a90b-5b4b2078f3cd None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:54.362176 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320ca0e-c5d1-4851-a90b-5b4b2078f3cd None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:54.362176 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2320ca0e-c5d1-4851-a90b-5b4b2078f3cd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:54.362407 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 423/854] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:54.362483 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2320ca0e-c5d1-4851-a90b-5b4b2078f3cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:54.362552 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa950363-7eb3-4419-ba03-e4d140305c65 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:54.366189 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2320ca0e-c5d1-4851-a90b-5b4b2078f3cd None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:54.372806 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 432/855] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 1 16:21:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:54.373990 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240841f-1a21-403a-94b2-abdcd3c2e3d2 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:54.374316 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240841f-1a21-403a-94b2-abdcd3c2e3d2 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:54.374596 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240841f-1a21-403a-94b2-abdcd3c2e3d2 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:54.377684 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 433/856] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:54.377684 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240841f-1a21-403a-94b2-abdcd3c2e3d2 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:54.455662 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:54.465765 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa950363-7eb3-4419-ba03-e4d140305c65 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:54.478763 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240841f-1a21-403a-94b2-abdcd3c2e3d2 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:54.499379 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd85f91-cfdf-4536-819f-4a6194be6aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:54.500003 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd85f91-cfdf-4536-819f-4a6194be6aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:54.500280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd85f91-cfdf-4536-819f-4a6194be6aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:54.500546 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd85f91-cfdf-4536-819f-4a6194be6aa8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:54.511562 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd85f91-cfdf-4536-819f-4a6194be6aa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:54.512593 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3944e0a2-ce2a-4c71-b35f-a26b558a97f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:54.512995 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd85f91-cfdf-4536-819f-4a6194be6aa8 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:54.513199 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944e0a2-ce2a-4c71-b35f-a26b558a97f3 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:54.513484 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944e0a2-ce2a-4c71-b35f-a26b558a97f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:54.513764 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944e0a2-ce2a-4c71-b35f-a26b558a97f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:54.514433 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3944e0a2-ce2a-4c71-b35f-a26b558a97f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:54.514745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3944e0a2-ce2a-4c71-b35f-a26b558a97f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:54.516096 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3944e0a2-ce2a-4c71-b35f-a26b558a97f3 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:54.526913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-ffd85f91-cfdf-4536-819f-4a6194be6aa8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:54.526913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:54.526913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:54.526913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:54.526913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:54.526913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:54.526913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:54.526913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:54.526913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:54.526913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:54.526913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:54.526913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:54.526913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:54.526913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:54.526913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:54.526913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:54.527840 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:54.527840 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:54.527840 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:54.527840 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:54.527840 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:54.527840 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:54.527840 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:54.527840 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:54.527840 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:54.527840 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:54.527840 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:54.527840 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:54.527840 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:54.527840 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:54.527840 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:54.527840 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:54.527840 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:54.528727 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:54.528727 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:54.528727 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:54.528904 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 434/857] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:54.528904 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3944e0a2-ce2a-4c71-b35f-a26b558a97f3 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:54.529026 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 424/858] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:54.529694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91ef13-75a7-418b-91e2-386734f7d02d None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:54.529971 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91ef13-75a7-418b-91e2-386734f7d02d None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:54.530246 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91ef13-75a7-418b-91e2-386734f7d02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:54.548305 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-2a91ef13-75a7-418b-91e2-386734f7d02d None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:54.549414 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2a91ef13-75a7-418b-91e2-386734f7d02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:54.576871 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:54.614075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 425/859] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:21:54.614929 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a91ef13-75a7-418b-91e2-386734f7d02d None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:54.641839 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3944e0a2-ce2a-4c71-b35f-a26b558a97f3 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:54.677322 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8758a-c3fc-4e99-959d-2aecc579841e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:54.677934 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8758a-c3fc-4e99-959d-2aecc579841e None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:54.678217 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8758a-c3fc-4e99-959d-2aecc579841e None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:54.678520 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8758a-c3fc-4e99-959d-2aecc579841e None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:54.679148 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba8758a-c3fc-4e99-959d-2aecc579841e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:54.679457 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba8758a-c3fc-4e99-959d-2aecc579841e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:54.680828 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba8758a-c3fc-4e99-959d-2aecc579841e None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:54.686072 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:54.688354 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 435/860] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 1 16:21:54 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:54.688973 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8758a-c3fc-4e99-959d-2aecc579841e None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:54.728910 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a91ef13-75a7-418b-91e2-386734f7d02d None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:54.770123 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270e235-2b11-49e9-af0d-551e7afaecc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:54.771185 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270e235-2b11-49e9-af0d-551e7afaecc9 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:54.771668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270e235-2b11-49e9-af0d-551e7afaecc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:54.772162 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270e235-2b11-49e9-af0d-551e7afaecc9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:54.780913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f270e235-2b11-49e9-af0d-551e7afaecc9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:54.782745 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f270e235-2b11-49e9-af0d-551e7afaecc9 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:54.796080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-f270e235-2b11-49e9-af0d-551e7afaecc9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:54.796080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:54.796080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:54.796080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:54.796080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:54.796080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:54.796080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:54.796080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:54.796080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:54.796080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:54.796080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:54.796080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:54.796080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:21:54.796080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:21:54.796080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:21:54.797033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:21:54.797033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:54.797033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:54.797033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:54.797033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:54.797033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:54.797033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:54.797033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:54.797033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:54.797033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:54.797033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:54.797033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:54.797033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:54.797033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:54.797033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:54.797033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:54.797033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:54.797901 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:21:54.797901 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:21:54.797901 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:21:54.797901 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:21:54.797901 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:21:54.797901 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:21:54.797901 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:54.797901 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:54.805253 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 426/861] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:54.806590 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270e235-2b11-49e9-af0d-551e7afaecc9 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:54.830781 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8758a-c3fc-4e99-959d-2aecc579841e None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:54.843731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:54.873843 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409a1f2-29d3-4b52-ae97-0bf11f904369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:54.874489 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409a1f2-29d3-4b52-ae97-0bf11f904369 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:54.874775 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409a1f2-29d3-4b52-ae97-0bf11f904369 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:54.875053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409a1f2-29d3-4b52-ae97-0bf11f904369 None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:54.875702 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409a1f2-29d3-4b52-ae97-0bf11f904369 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:54.876008 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409a1f2-29d3-4b52-ae97-0bf11f904369 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:54.877392 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409a1f2-29d3-4b52-ae97-0bf11f904369 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:54.890634 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 436/862] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:54.891475 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409a1f2-29d3-4b52-ae97-0bf11f904369 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:54.933161 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270e235-2b11-49e9-af0d-551e7afaecc9 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:54.967554 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:54.973898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55521dcd-50ec-46f3-a155-ed086ed81138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:54.974960 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55521dcd-50ec-46f3-a155-ed086ed81138 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:54.975247 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55521dcd-50ec-46f3-a155-ed086ed81138 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:54.975521 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55521dcd-50ec-46f3-a155-ed086ed81138 None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:54.976584 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55521dcd-50ec-46f3-a155-ed086ed81138 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:54.976898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55521dcd-50ec-46f3-a155-ed086ed81138 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:54.979250 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55521dcd-50ec-46f3-a155-ed086ed81138 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:54.991650 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 427/863] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:21:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:54.992273 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55521dcd-50ec-46f3-a155-ed086ed81138 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:55.005570 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409a1f2-29d3-4b52-ae97-0bf11f904369 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:55.043547 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab2923-4ce5-4233-be5d-91f6a9a4cbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:55.044623 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab2923-4ce5-4233-be5d-91f6a9a4cbf0 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:55.045121 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab2923-4ce5-4233-be5d-91f6a9a4cbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:55.045583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab2923-4ce5-4233-be5d-91f6a9a4cbf0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:55.046425 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ab2923-4ce5-4233-be5d-91f6a9a4cbf0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:55.048061 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ab2923-4ce5-4233-be5d-91f6a9a4cbf0 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:55.057762 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-40ab2923-4ce5-4233-be5d-91f6a9a4cbf0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:21:55.057762 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:55.057762 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:21:55.057762 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:21:55.057762 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:21:55.057762 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:21:55.057762 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:21:55.057762 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:21:55.057762 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:21:55.057762 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:55.057762 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:21:55.057762 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:55.057762 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:55.057762 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:55.057762 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:55.057762 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:55.058716 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:55.058716 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:55.058716 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:55.058716 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:55.058716 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:55.058716 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:55.058716 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:55.058716 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:21:55.058716 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:21:55.058716 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:21:55.058716 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:21:55.058716 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:55.058716 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:55.058716 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:55.058716 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:55.058716 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:55.058716 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:55.059602 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:55.059602 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:55.059602 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:55.059602 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:55.059602 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:55.059602 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:55.059602 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:55.059602 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:55.059602 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:55.059602 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:55.059602 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:21:55.059602 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:21:55.059602 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:21:55.059602 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:55.063608 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 437/864] 10.222.0.18 () {56 vars in 1132 bytes} [Thu Sep 1 16:21:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:55.064394 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab2923-4ce5-4233-be5d-91f6a9a4cbf0 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:55.095835 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55521dcd-50ec-46f3-a155-ed086ed81138 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:55.097827 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:55.132367 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70472e1-ea1b-4081-b179-a64b13c8fe47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:55.133205 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70472e1-ea1b-4081-b179-a64b13c8fe47 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:55.133514 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70472e1-ea1b-4081-b179-a64b13c8fe47 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:55.133799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70472e1-ea1b-4081-b179-a64b13c8fe47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:55.141835 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70472e1-ea1b-4081-b179-a64b13c8fe47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:55.143311 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70472e1-ea1b-4081-b179-a64b13c8fe47 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:55.155653 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-f70472e1-ea1b-4081-b179-a64b13c8fe47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:55.155653 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:55.155653 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:55.155653 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:55.155653 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:55.155653 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:55.155653 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:55.155653 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:55.155653 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:55.155653 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:55.155653 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:55.155653 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:55.155653 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:21:55.155653 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:21:55.155653 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:21:55.156723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:21:55.156723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:55.156723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:55.156723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:55.156723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:55.156723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:55.156723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:55.156723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:55.156723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:55.156723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:55.156723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:55.156723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:55.156723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:55.156723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:55.156723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:55.156723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:55.156723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:55.157595 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:21:55.157595 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:21:55.157595 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:21:55.157595 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:21:55.157595 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:21:55.157595 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:21:55.157595 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:55.157595 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:55.157595 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 428/865] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:55.158073 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70472e1-ea1b-4081-b179-a64b13c8fe47 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:55.174183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab2923-4ce5-4233-be5d-91f6a9a4cbf0 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:55.208762 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e95f7-acbf-4782-a819-0dd235d3873e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:55.209377 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e95f7-acbf-4782-a819-0dd235d3873e None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:55.209659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e95f7-acbf-4782-a819-0dd235d3873e None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:55.209935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e95f7-acbf-4782-a819-0dd235d3873e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:55.210691 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863e95f7-acbf-4782-a819-0dd235d3873e None admin] RBAC: Adding query filter params (role.id=e8ba063905484d22993bcf04830091a3, scope.project.id=36ce80c4e0d64562a9ab3961be76fad9, user.id=8137f6b2ced040bd9752239c7e987f03) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:55.211003 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863e95f7-acbf-4782-a819-0dd235d3873e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:55.212997 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863e95f7-acbf-4782-a819-0dd235d3873e None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:55.218986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 438/866] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:21:55 2022] GET /identity/v3/role_assignments?user.id=8137f6b2ced040bd9752239c7e987f03&scope.project.id=36ce80c4e0d64562a9ab3961be76fad9&role.id=e8ba063905484d22993bcf04830091a3 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:55.219743 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e95f7-acbf-4782-a819-0dd235d3873e None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:55.231248 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:55.292143 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70472e1-ea1b-4081-b179-a64b13c8fe47 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:55.336070 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de8063-2002-4244-b82c-32efdd9f0567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:55.336771 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de8063-2002-4244-b82c-32efdd9f0567 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:55.337152 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de8063-2002-4244-b82c-32efdd9f0567 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:55.338138 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de8063-2002-4244-b82c-32efdd9f0567 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:55.338905 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08de8063-2002-4244-b82c-32efdd9f0567 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:55.339308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08de8063-2002-4244-b82c-32efdd9f0567 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:55.340795 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08de8063-2002-4244-b82c-32efdd9f0567 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:55.353637 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 429/867] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Sep 1 16:21:55 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:55.353637 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de8063-2002-4244-b82c-32efdd9f0567 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:55.356396 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:55.361407 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e95f7-acbf-4782-a819-0dd235d3873e None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:55.408748 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c88af-b27d-465c-b6a1-933b9bfbf9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:55.409366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c88af-b27d-465c-b6a1-933b9bfbf9de None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:55.409651 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c88af-b27d-465c-b6a1-933b9bfbf9de None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:55.409925 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c88af-b27d-465c-b6a1-933b9bfbf9de None admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:55.410656 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544c88af-b27d-465c-b6a1-933b9bfbf9de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:55.410969 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544c88af-b27d-465c-b6a1-933b9bfbf9de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:55.412412 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544c88af-b27d-465c-b6a1-933b9bfbf9de None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:55.425718 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 439/868] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:21:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:55.431437 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c88af-b27d-465c-b6a1-933b9bfbf9de None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:55.471074 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de8063-2002-4244-b82c-32efdd9f0567 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:55.489589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:55.511227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d368babc-59ae-4b90-8872-cb9d7ad51eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:55.511872 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368babc-59ae-4b90-8872-cb9d7ad51eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:55.512155 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368babc-59ae-4b90-8872-cb9d7ad51eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:55.512433 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368babc-59ae-4b90-8872-cb9d7ad51eb0 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:55.513069 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d368babc-59ae-4b90-8872-cb9d7ad51eb0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:55.513380 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d368babc-59ae-4b90-8872-cb9d7ad51eb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:55.514771 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d368babc-59ae-4b90-8872-cb9d7ad51eb0 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:55.519735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 430/869] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Sep 1 16:21:55 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:55.553527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c88af-b27d-465c-b6a1-933b9bfbf9de None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:55.593892 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b24da5-181d-479b-b46d-378a10204977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:55.594795 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b24da5-181d-479b-b46d-378a10204977 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:55.595260 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b24da5-181d-479b-b46d-378a10204977 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:55.595773 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b24da5-181d-479b-b46d-378a10204977 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:55.596680 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b24da5-181d-479b-b46d-378a10204977 None admin] RBAC: Adding query filter params (role.id=b6727ed962fa41109b0a2d6851551f49, scope.project.id=36ce80c4e0d64562a9ab3961be76fad9, user.id=a5b3e53fd1314e17ad28c42a44e8eff3) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:55.597145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b24da5-181d-479b-b46d-378a10204977 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:55.598825 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b24da5-181d-479b-b46d-378a10204977 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:55.606168 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 440/870] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:21:55 2022] GET /identity/v3/role_assignments?user.id=a5b3e53fd1314e17ad28c42a44e8eff3&scope.project.id=36ce80c4e0d64562a9ab3961be76fad9&role.id=b6727ed962fa41109b0a2d6851551f49 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:21:56.649846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c0e26-df0c-4b16-befb-e8bcacabdc91 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:56.650830 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c0e26-df0c-4b16-befb-e8bcacabdc91 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:56.650830 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c0e26-df0c-4b16-befb-e8bcacabdc91 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:56.652648 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 431/871] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:56.657794 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ccfb6-f914-4070-a3b1-bf63b6179f34 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:56.658211 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ccfb6-f914-4070-a3b1-bf63b6179f34 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:56.658373 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ccfb6-f914-4070-a3b1-bf63b6179f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:56.676504 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-924ccfb6-f914-4070-a3b1-bf63b6179f34 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:56.677696 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-924ccfb6-f914-4070-a3b1-bf63b6179f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:56.694651 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:56.752087 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 441/872] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:21:57.022571 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c0e26-df0c-4b16-befb-e8bcacabdc91 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:57.135557 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c0e26-df0c-4b16-befb-e8bcacabdc91 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:57.171207 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bdef09-7113-4aa3-b31a-4875ef07beea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:57.171824 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bdef09-7113-4aa3-b31a-4875ef07beea None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:57.172107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bdef09-7113-4aa3-b31a-4875ef07beea None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:57.172383 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bdef09-7113-4aa3-b31a-4875ef07beea None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:57.173044 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bdef09-7113-4aa3-b31a-4875ef07beea None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:57.174437 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bdef09-7113-4aa3-b31a-4875ef07beea None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:57.193113 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 432/873] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Sep 1 16:21:57 2022] POST /identity/v3/services => generated 251 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:21:57.272160 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd6610-5bbf-486c-8698-100c27f23fb8 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:57.272707 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd6610-5bbf-486c-8698-100c27f23fb8 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:57.273038 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd6610-5bbf-486c-8698-100c27f23fb8 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:57.275408 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 442/874] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:21:57.281431 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0fe8a-eedf-43bb-8806-8db20522660b None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:57.281722 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0fe8a-eedf-43bb-8806-8db20522660b None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:57.282091 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0fe8a-eedf-43bb-8806-8db20522660b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:57.302676 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-07e0fe8a-eedf-43bb-8806-8db20522660b None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:57.303976 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-07e0fe8a-eedf-43bb-8806-8db20522660b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:57.343686 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:57.390773 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 433/875] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:57 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:21:57.484147 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fded8e-0399-4d72-a323-7496734a4319 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:57.484147 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fded8e-0399-4d72-a323-7496734a4319 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:57.484147 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fded8e-0399-4d72-a323-7496734a4319 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:57.484790 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 443/876] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:57.490057 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f287c-fd19-4e2b-ae52-e9309e87dddd None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:57.490578 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f287c-fd19-4e2b-ae52-e9309e87dddd None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:57.491042 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f287c-fd19-4e2b-ae52-e9309e87dddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:57.510694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-7b3f287c-fd19-4e2b-ae52-e9309e87dddd None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:57.512139 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7b3f287c-fd19-4e2b-ae52-e9309e87dddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:57.580945 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 434/877] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:57 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:21:57.612875 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ea096-24a4-46e0-9ab5-13cdfe873d6c None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:57.613514 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ea096-24a4-46e0-9ab5-13cdfe873d6c None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:57.614007 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ea096-24a4-46e0-9ab5-13cdfe873d6c None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:57.615817 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 444/878] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:57.621388 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa60b7-3b12-44e3-8c94-12d8efecc445 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:57.621941 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa60b7-3b12-44e3-8c94-12d8efecc445 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:57.622462 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa60b7-3b12-44e3-8c94-12d8efecc445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:57.641250 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-4baa60b7-3b12-44e3-8c94-12d8efecc445 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:57.642740 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-4baa60b7-3b12-44e3-8c94-12d8efecc445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:57.661182 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ea096-24a4-46e0-9ab5-13cdfe873d6c None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:57.717894 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 435/879] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:57 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:21:57.770329 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ea096-24a4-46e0-9ab5-13cdfe873d6c None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:57.804876 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda4557-56ed-49ee-aadd-4041ea0ffad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:57.805511 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda4557-56ed-49ee-aadd-4041ea0ffad0 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:57.805794 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda4557-56ed-49ee-aadd-4041ea0ffad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:57.806662 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda4557-56ed-49ee-aadd-4041ea0ffad0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:57.807289 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda4557-56ed-49ee-aadd-4041ea0ffad0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:57.808767 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda4557-56ed-49ee-aadd-4041ea0ffad0 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:57.817836 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-3fda4557-56ed-49ee-aadd-4041ea0ffad0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:21:57.817836 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:57.817836 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:21:57.817836 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:21:57.817836 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:21:57.817836 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:21:57.817836 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:21:57.817836 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:21:57.817836 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:21:57.817836 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:57.817836 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:21:57.817836 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:57.817836 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:57.817836 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:57.817836 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:57.817836 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:57.817836 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:57.818821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:57.818821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:57.818821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:57.818821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:57.818821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:57.818821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:57.818821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:21:57.818821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:21:57.818821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:21:57.818821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:21:57.818821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:57.818821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:57.818821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:57.818821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:57.818821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:57.818821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:57.819635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:57.819635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:57.819635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:57.819635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:57.819635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:57.819635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:57.819635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:57.819635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:57.819635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:57.819635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:57.819635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:21:57.819635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:21:57.819635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:21:57.819635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:57.819635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 445/880] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Sep 1 16:21:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:21:57.827609 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baa60b7-3b12-44e3-8c94-12d8efecc445 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:57.855881 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:57.870229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda4557-56ed-49ee-aadd-4041ea0ffad0 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:57.889500 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:57.948439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baa60b7-3b12-44e3-8c94-12d8efecc445 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:57.988500 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda4557-56ed-49ee-aadd-4041ea0ffad0 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:57.990363 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c0b77-6a41-4dc7-b7da-0c60d6e3e276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:57.991006 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c0b77-6a41-4dc7-b7da-0c60d6e3e276 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:57.991287 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c0b77-6a41-4dc7-b7da-0c60d6e3e276 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:57.991562 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c0b77-6a41-4dc7-b7da-0c60d6e3e276 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:57.992186 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6c0b77-6a41-4dc7-b7da-0c60d6e3e276 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:57.992488 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6c0b77-6a41-4dc7-b7da-0c60d6e3e276 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:57.994431 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6c0b77-6a41-4dc7-b7da-0c60d6e3e276 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:57.999677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 436/881] 10.222.0.18 () {56 vars in 1122 bytes} [Thu Sep 1 16:21:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:58.000526 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c0b77-6a41-4dc7-b7da-0c60d6e3e276 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:58.027612 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90255850-df4a-421d-a085-c989246b4c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:58.028221 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90255850-df4a-421d-a085-c989246b4c9c None admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:58.028508 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90255850-df4a-421d-a085-c989246b4c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:58.028791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90255850-df4a-421d-a085-c989246b4c9c None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:58.029440 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90255850-df4a-421d-a085-c989246b4c9c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:58.030861 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90255850-df4a-421d-a085-c989246b4c9c None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:58.047779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 446/882] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Sep 1 16:21:57 2022] POST /identity/v3/services => generated 269 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:21:58.048591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90255850-df4a-421d-a085-c989246b4c9c None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:58.104920 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:58.117824 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c0b77-6a41-4dc7-b7da-0c60d6e3e276 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:58.168091 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5e29c-bf93-47b5-aa56-93f2c048d2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:58.169104 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5e29c-bf93-47b5-aa56-93f2c048d2c9 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:58.169559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5e29c-bf93-47b5-aa56-93f2c048d2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:58.170019 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5e29c-bf93-47b5-aa56-93f2c048d2c9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:58.176349 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:58.186670 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d5e29c-bf93-47b5-aa56-93f2c048d2c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:21:58.192365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-92d5e29c-bf93-47b5-aa56-93f2c048d2c9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:21:58.192365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:58.192365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:58.192365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:58.192365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:58.192365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:58.192365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:58.192365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:58.192365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:58.192365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:58.192365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:58.192365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:58.192365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:21:58.192365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:21:58.192365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:21:58.192365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:21:58.193280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:21:58.193280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:21:58.193280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:21:58.193280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:21:58.193280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:58.193280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:58.193280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:58.193280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:58.193280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:58.193280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:58.193280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:58.193280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:58.193280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:58.193280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:58.193280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:58.193280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:58.194076 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:58.194076 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:58.194076 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:58.194076 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:58.194076 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:21:58.194076 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:21:58.194076 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:21:58.194076 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:21:58.194076 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:21:58.194076 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:21:58.194076 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:21:58.194076 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:58.197533 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 437/883] 10.222.0.18 () {56 vars in 1120 bytes} [Thu Sep 1 16:21:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:58.203917 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5e29c-bf93-47b5-aa56-93f2c048d2c9 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:58.204509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90255850-df4a-421d-a085-c989246b4c9c None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:58.252933 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee1c5e-1af5-41ff-b2ef-ddcc4fe08500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:58.253556 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee1c5e-1af5-41ff-b2ef-ddcc4fe08500 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:58.253835 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee1c5e-1af5-41ff-b2ef-ddcc4fe08500 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:58.254113 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee1c5e-1af5-41ff-b2ef-ddcc4fe08500 None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:58.254777 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee1c5e-1af5-41ff-b2ef-ddcc4fe08500 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:58.255082 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee1c5e-1af5-41ff-b2ef-ddcc4fe08500 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:58.256452 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee1c5e-1af5-41ff-b2ef-ddcc4fe08500 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:58.262092 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 447/884] 10.222.0.18 () {56 vars in 1122 bytes} [Thu Sep 1 16:21:58 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:58.332217 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5e29c-bf93-47b5-aa56-93f2c048d2c9 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:58.372560 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311d7faf-3b25-4308-bc9b-5ed2ad5adee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:58.373400 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d7faf-3b25-4308-bc9b-5ed2ad5adee9 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:58.373864 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d7faf-3b25-4308-bc9b-5ed2ad5adee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:58.374312 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d7faf-3b25-4308-bc9b-5ed2ad5adee9 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:58.375139 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311d7faf-3b25-4308-bc9b-5ed2ad5adee9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:58.375594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311d7faf-3b25-4308-bc9b-5ed2ad5adee9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:58.377112 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311d7faf-3b25-4308-bc9b-5ed2ad5adee9 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:58.387870 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 438/885] 10.222.0.18 () {56 vars in 1115 bytes} [Thu Sep 1 16:21:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:58.398292 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee1c5e-1af5-41ff-b2ef-ddcc4fe08500 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:58.503853 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee1c5e-1af5-41ff-b2ef-ddcc4fe08500 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:58.539156 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958f1ea-b937-44a5-ad44-f9b7b098fbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:58.540088 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958f1ea-b937-44a5-ad44-f9b7b098fbf1 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:58.541103 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958f1ea-b937-44a5-ad44-f9b7b098fbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:58.541532 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958f1ea-b937-44a5-ad44-f9b7b098fbf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:58.550787 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:58.553510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958f1ea-b937-44a5-ad44-f9b7b098fbf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:58.555537 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958f1ea-b937-44a5-ad44-f9b7b098fbf1 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:58.570772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-7958f1ea-b937-44a5-ad44-f9b7b098fbf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:58.570772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:58.570772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:58.570772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:58.570772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:58.570772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:58.570772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:58.570772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:58.570772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:58.570772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:58.570772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:58.570772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:58.570772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:58.570772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:58.570772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:58.570772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:58.571986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:58.571986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:58.571986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:58.571986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:58.571986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:58.571986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:58.571986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:58.571986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:58.571986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:58.571986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:58.571986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:58.571986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:58.571986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:58.571986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:58.571986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:58.571986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:58.571986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:58.573081 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:58.573081 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:58.573081 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:58.574796 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 448/886] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:58.580312 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311d7faf-3b25-4308-bc9b-5ed2ad5adee9 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:58.595727 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:58.693170 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311d7faf-3b25-4308-bc9b-5ed2ad5adee9 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:58.729869 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8951806-496b-47fa-8fba-47cdaa9ef43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:58.731244 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8951806-496b-47fa-8fba-47cdaa9ef43a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:58.731780 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8951806-496b-47fa-8fba-47cdaa9ef43a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:58.732239 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8951806-496b-47fa-8fba-47cdaa9ef43a None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:58.733079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8951806-496b-47fa-8fba-47cdaa9ef43a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:58.733568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8951806-496b-47fa-8fba-47cdaa9ef43a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:58.735196 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8951806-496b-47fa-8fba-47cdaa9ef43a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:58.748847 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 439/887] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:58.754464 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958f1ea-b937-44a5-ad44-f9b7b098fbf1 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:58.871280 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958f1ea-b937-44a5-ad44-f9b7b098fbf1 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:58.907430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e3c68a-1cde-4225-a4d0-2a8395d73604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:58.908073 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3c68a-1cde-4225-a4d0-2a8395d73604 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:58.908357 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3c68a-1cde-4225-a4d0-2a8395d73604 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:58.908631 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3c68a-1cde-4225-a4d0-2a8395d73604 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:58.917276 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e3c68a-1cde-4225-a4d0-2a8395d73604 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:58.918767 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e3c68a-1cde-4225-a4d0-2a8395d73604 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:58.930070 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-53e3c68a-1cde-4225-a4d0-2a8395d73604 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:21:58.930070 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:58.930070 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:58.930070 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:58.930070 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:58.930070 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:58.930070 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:58.930070 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:58.930070 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:58.930070 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:58.930070 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:58.930070 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:58.930070 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:21:58.930070 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:21:58.930070 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:21:58.930070 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:21:58.931056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:58.931056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:58.931056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:21:58.931056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:58.931056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:21:58.931056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:21:58.931056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:58.931056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:58.931056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:58.931056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:58.931056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:58.931056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:58.931056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:58.931056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:58.931056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:58.931056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:58.931056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:58.931056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:58.931980 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:58.931980 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:58.931980 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:21:58.931980 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:21:58.931980 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:21:58.931980 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:21:58.931980 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:21:58.931980 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:21:58.931980 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:21:58.931980 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:58.931980 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 449/888] 10.222.0.18 () {56 vars in 1117 bytes} [Thu Sep 1 16:21:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:58.937090 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8951806-496b-47fa-8fba-47cdaa9ef43a None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:59.014623 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:59.050976 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8951806-496b-47fa-8fba-47cdaa9ef43a None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:59.087818 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26676b43-d4f0-4746-93c2-ac625acf6fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:59.088783 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26676b43-d4f0-4746-93c2-ac625acf6fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:59.089272 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26676b43-d4f0-4746-93c2-ac625acf6fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:59.089782 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26676b43-d4f0-4746-93c2-ac625acf6fd7 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:59.090762 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26676b43-d4f0-4746-93c2-ac625acf6fd7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:59.091281 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26676b43-d4f0-4746-93c2-ac625acf6fd7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:59.092906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26676b43-d4f0-4746-93c2-ac625acf6fd7 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:59.112232 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 440/889] 10.222.0.18 () {56 vars in 1149 bytes} [Thu Sep 1 16:21:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:59.122291 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e3c68a-1cde-4225-a4d0-2a8395d73604 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:59.175463 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:59.227568 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e3c68a-1cde-4225-a4d0-2a8395d73604 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:59.228869 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71d372-641a-490e-b5c6-655612228455 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:59.229256 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71d372-641a-490e-b5c6-655612228455 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:59.229534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71d372-641a-490e-b5c6-655612228455 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:59.232186 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 441/890] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:21:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:21:59.236180 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b92d8-a9be-4833-8c9a-fa7003008559 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:59.236466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b92d8-a9be-4833-8c9a-fa7003008559 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:59.236741 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b92d8-a9be-4833-8c9a-fa7003008559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:59.256280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-7c5b92d8-a9be-4833-8c9a-fa7003008559 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:21:59.257548 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7c5b92d8-a9be-4833-8c9a-fa7003008559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:21:59.265106 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd0e3f-5b6e-47aa-962f-ab1368f24c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:59.265669 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd0e3f-5b6e-47aa-962f-ab1368f24c33 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:59.265959 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd0e3f-5b6e-47aa-962f-ab1368f24c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:59.278279 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd0e3f-5b6e-47aa-962f-ab1368f24c33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:59.333686 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 442/891] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:21:59 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:21:59.336908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfd0e3f-5b6e-47aa-962f-ab1368f24c33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:59.338444 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfd0e3f-5b6e-47aa-962f-ab1368f24c33 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:59.398752 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:59.449449 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 450/892] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:21:59 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:21:59.455192 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b92d8-a9be-4833-8c9a-fa7003008559 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:59.519679 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:59.564920 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b92d8-a9be-4833-8c9a-fa7003008559 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:59.600336 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d6713-5301-4238-b4df-817ea1cad296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:59.600977 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d6713-5301-4238-b4df-817ea1cad296 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:59.601269 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d6713-5301-4238-b4df-817ea1cad296 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:59.601545 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d6713-5301-4238-b4df-817ea1cad296 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:59.611981 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d6713-5301-4238-b4df-817ea1cad296 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:59.613489 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d6713-5301-4238-b4df-817ea1cad296 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:59.616789 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd0e3f-5b6e-47aa-962f-ab1368f24c33 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:59.629687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-b60d6713-5301-4238-b4df-817ea1cad296 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:59.629687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:59.629687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:59.629687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:59.629687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:59.629687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:59.629687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:59.629687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:59.629687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:59.629687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:59.629687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:59.629687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:59.629687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:21:59.629687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:21:59.629687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:21:59.629687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:21:59.630669 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:59.630669 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:59.630669 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:59.630669 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:59.630669 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:59.630669 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:59.630669 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:59.630669 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:59.630669 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:59.630669 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:59.630669 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:59.630669 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:59.630669 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:59.630669 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:59.630669 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:59.630669 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:59.630669 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:21:59.631534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:21:59.631534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:21:59.631534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:21:59.631534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 443/893] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:21:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:21:59.642493 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d6713-5301-4238-b4df-817ea1cad296 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:59.751185 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd0e3f-5b6e-47aa-962f-ab1368f24c33 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:59.757918 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d6713-5301-4238-b4df-817ea1cad296 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:59.768120 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:59.795643 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e83d9-eb7d-4665-b38c-dc7f7da873e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:59.796368 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e83d9-eb7d-4665-b38c-dc7f7da873e4 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:59.796723 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e83d9-eb7d-4665-b38c-dc7f7da873e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:59.797167 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e83d9-eb7d-4665-b38c-dc7f7da873e4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:59.797828 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dcd23a-97d8-44d3-b3fa-cd66503831df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:59.797949 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78e83d9-eb7d-4665-b38c-dc7f7da873e4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:59.798748 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dcd23a-97d8-44d3-b3fa-cd66503831df None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:59.799229 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dcd23a-97d8-44d3-b3fa-cd66503831df None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:59.799409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78e83d9-eb7d-4665-b38c-dc7f7da873e4 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:59.799943 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dcd23a-97d8-44d3-b3fa-cd66503831df None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:59.800837 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dcd23a-97d8-44d3-b3fa-cd66503831df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:59.801321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dcd23a-97d8-44d3-b3fa-cd66503831df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:59.802896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dcd23a-97d8-44d3-b3fa-cd66503831df None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:59.810307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-f78e83d9-eb7d-4665-b38c-dc7f7da873e4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:21:59.810307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:59.810307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:21:59.810307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:21:59.810307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:21:59.810307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:21:59.810307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:21:59.810307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:21:59.810307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:21:59.810307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:59.810307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:21:59.810307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:59.810307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:59.810307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:59.810307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:59.810307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:59.810307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:59.811288 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:59.811288 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:59.811288 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:59.811288 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:59.811288 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:59.811288 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:59.811288 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:21:59.811288 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:21:59.811288 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:21:59.811288 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:21:59.811288 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:59.811288 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:59.811288 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:59.811288 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:59.811288 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:59.811288 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:59.812172 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:59.812172 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:59.812172 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:59.812172 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:59.812172 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:59.812172 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:59.812172 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:59.812172 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:59.812172 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:59.812172 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:21:59.812172 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:21:59.812172 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:21:59.812172 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:21:59.812172 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:21:59.812172 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 451/894] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 1 16:21:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:21:59.817549 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e83d9-eb7d-4665-b38c-dc7f7da873e4 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:59.824411 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 444/895] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:21:59.825524 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dcd23a-97d8-44d3-b3fa-cd66503831df None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:21:59.840747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:21:59.935761 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dcd23a-97d8-44d3-b3fa-cd66503831df None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:59.949040 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e83d9-eb7d-4665-b38c-dc7f7da873e4 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:21:59.972597 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5206e828-c575-40c5-b51c-87a03b988275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:59.973536 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206e828-c575-40c5-b51c-87a03b988275 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:59.974001 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206e828-c575-40c5-b51c-87a03b988275 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:59.974488 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206e828-c575-40c5-b51c-87a03b988275 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:59.982398 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5206e828-c575-40c5-b51c-87a03b988275 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:59.984278 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5206e828-c575-40c5-b51c-87a03b988275 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:59.986735 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d69e5e-025e-411d-b8fe-0078b1cd86f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:21:59.987602 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d69e5e-025e-411d-b8fe-0078b1cd86f1 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:21:59.987715 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d69e5e-025e-411d-b8fe-0078b1cd86f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:21:59.988067 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d69e5e-025e-411d-b8fe-0078b1cd86f1 None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:21:59.988773 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d69e5e-025e-411d-b8fe-0078b1cd86f1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:21:59.989152 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d69e5e-025e-411d-b8fe-0078b1cd86f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:21:59.990618 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d69e5e-025e-411d-b8fe-0078b1cd86f1 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:21:59.996797 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 452/896] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 1 16:21:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:21:59.998883 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-5206e828-c575-40c5-b51c-87a03b988275 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:21:59.998883 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:21:59.998883 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:21:59.998883 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:21:59.998883 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:21:59.998883 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:21:59.998883 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:21:59.998883 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:21:59.998883 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:21:59.998883 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:21:59.998883 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:21:59.998883 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:21:59.998883 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:21:59.998883 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:21:59.998883 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:21:59.999715 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:21:59.999715 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:21:59.999715 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:21:59.999715 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:21:59.999715 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:21:59.999715 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:21:59.999715 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:21:59.999715 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:21:59.999715 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:21:59.999715 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:21:59.999715 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:21:59.999715 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:21:59.999715 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:21:59.999715 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:21:59.999715 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:21:59.999715 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:21:59.999715 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:00.000588 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:22:00.000588 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:22:00.000588 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:22:00.000588 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:22:00.000588 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:22:00.000588 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:22:00.000588 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:22:00.000588 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:00.006924 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 445/897] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:21:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:00.007681 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d69e5e-025e-411d-b8fe-0078b1cd86f1 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:00.008057 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5206e828-c575-40c5-b51c-87a03b988275 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:00.039454 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:00.096118 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:00.116279 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5206e828-c575-40c5-b51c-87a03b988275 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:00.116729 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d69e5e-025e-411d-b8fe-0078b1cd86f1 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:00.150390 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f112e6d-fc4e-42c5-9288-4ad7f8018a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:00.151212 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b6a08-54ac-4a42-9539-94fed4445b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:00.151357 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f112e6d-fc4e-42c5-9288-4ad7f8018a9f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:00.151357 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f112e6d-fc4e-42c5-9288-4ad7f8018a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:00.151530 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b6a08-54ac-4a42-9539-94fed4445b4a None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:00.151599 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f112e6d-fc4e-42c5-9288-4ad7f8018a9f None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:00.151674 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b6a08-54ac-4a42-9539-94fed4445b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:00.151991 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b6a08-54ac-4a42-9539-94fed4445b4a None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:00.152345 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f112e6d-fc4e-42c5-9288-4ad7f8018a9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:00.152622 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739b6a08-54ac-4a42-9539-94fed4445b4a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:00.152725 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f112e6d-fc4e-42c5-9288-4ad7f8018a9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:00.152932 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739b6a08-54ac-4a42-9539-94fed4445b4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:00.154074 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f112e6d-fc4e-42c5-9288-4ad7f8018a9f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:00.154351 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739b6a08-54ac-4a42-9539-94fed4445b4a None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:00.164399 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 453/898] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 1 16:22:00 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:00.164399 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f130e75-2b35-4c2b-a9e9-486dffde790e None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:00.164399 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f130e75-2b35-4c2b-a9e9-486dffde790e None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:00.164399 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f130e75-2b35-4c2b-a9e9-486dffde790e None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:00.164399 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 454/899] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:00.164963 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f130e75-2b35-4c2b-a9e9-486dffde790e None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:00.176422 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 446/900] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:22:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:00.176422 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1f074-439d-4d9b-ad5f-4160a15a18b2 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:00.176422 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1f074-439d-4d9b-ad5f-4160a15a18b2 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:00.176422 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1f074-439d-4d9b-ad5f-4160a15a18b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:00.191486 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-79a1f074-439d-4d9b-ad5f-4160a15a18b2 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:00.192685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-79a1f074-439d-4d9b-ad5f-4160a15a18b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:00.270272 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 447/901] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:00 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:22:00.270272 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a1f074-439d-4d9b-ad5f-4160a15a18b2 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:00.284690 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f130e75-2b35-4c2b-a9e9-486dffde790e None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:00.310877 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:00.322419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606e3e0-8b10-4180-bc78-d3720c3f6706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:00.323212 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606e3e0-8b10-4180-bc78-d3720c3f6706 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:00.323601 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606e3e0-8b10-4180-bc78-d3720c3f6706 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:00.323987 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606e3e0-8b10-4180-bc78-d3720c3f6706 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:00.324791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606e3e0-8b10-4180-bc78-d3720c3f6706 None admin] RBAC: Adding query filter params (interface=public, service_id=e0bcbfca77e144c0b6ad1f312d742825, region_id=RegionOne) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:00.325208 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606e3e0-8b10-4180-bc78-d3720c3f6706 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:00.326819 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606e3e0-8b10-4180-bc78-d3720c3f6706 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:00.333039 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 455/902] 10.222.0.18 () {56 vars in 1251 bytes} [Thu Sep 1 16:22:00 2022] GET /identity/v3/endpoints?service_id=e0bcbfca77e144c0b6ad1f312d742825&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:00.338466 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606e3e0-8b10-4180-bc78-d3720c3f6706 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:00.353311 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:00.401193 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a1f074-439d-4d9b-ad5f-4160a15a18b2 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:00.439568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb644eb-b8ff-4f20-b412-a7a37262ba23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:00.440182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb644eb-b8ff-4f20-b412-a7a37262ba23 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:00.440478 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb644eb-b8ff-4f20-b412-a7a37262ba23 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:00.440746 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb644eb-b8ff-4f20-b412-a7a37262ba23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:00.441472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb644eb-b8ff-4f20-b412-a7a37262ba23 None admin] RBAC: Adding query filter params (role.id=a889b69245b44da59d9020a0d39751cb, scope.project.id=36ce80c4e0d64562a9ab3961be76fad9, user.id=a5b3e53fd1314e17ad28c42a44e8eff3) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:00.441791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb644eb-b8ff-4f20-b412-a7a37262ba23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:00.442067 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606e3e0-8b10-4180-bc78-d3720c3f6706 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:00.443357 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb644eb-b8ff-4f20-b412-a7a37262ba23 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:00.449692 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 448/903] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:22:00 2022] GET /identity/v3/role_assignments?user.id=a5b3e53fd1314e17ad28c42a44e8eff3&scope.project.id=36ce80c4e0d64562a9ab3961be76fad9&role.id=a889b69245b44da59d9020a0d39751cb => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:00.450942 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dff6ea-8c70-4781-9f5d-d8d99bcf6f52 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:00.451231 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dff6ea-8c70-4781-9f5d-d8d99bcf6f52 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:00.451509 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dff6ea-8c70-4781-9f5d-d8d99bcf6f52 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:00.456719 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 449/904] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:00.458279 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111c709-4030-44e1-bcd8-2366ae1f2af1 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:00.458569 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111c709-4030-44e1-bcd8-2366ae1f2af1 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:00.458975 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111c709-4030-44e1-bcd8-2366ae1f2af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:00.480043 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-d111c709-4030-44e1-bcd8-2366ae1f2af1 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:00.481275 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d111c709-4030-44e1-bcd8-2366ae1f2af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:00.499023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e263e99-5c5b-4a8d-a1f2-0febc450f2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:00.500567 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e263e99-5c5b-4a8d-a1f2-0febc450f2ed None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:00.500854 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e263e99-5c5b-4a8d-a1f2-0febc450f2ed None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:00.501127 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e263e99-5c5b-4a8d-a1f2-0febc450f2ed None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:00.501829 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e263e99-5c5b-4a8d-a1f2-0febc450f2ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:00.503421 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e263e99-5c5b-4a8d-a1f2-0febc450f2ed None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:00.510869 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 456/905] 10.222.0.18 () {56 vars in 1087 bytes} [Thu Sep 1 16:22:00 2022] GET /identity/v3/services => generated 1268 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:22:00.546785 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e263e99-5c5b-4a8d-a1f2-0febc450f2ed None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:00.586989 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 450/906] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:00 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:22:00.656222 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:00.689970 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e263e99-5c5b-4a8d-a1f2-0febc450f2ed None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:00.725693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5af7d2-f838-4c85-9bd4-1c49012d0214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:00.726361 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5af7d2-f838-4c85-9bd4-1c49012d0214 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:00.726644 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5af7d2-f838-4c85-9bd4-1c49012d0214 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:00.726915 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5af7d2-f838-4c85-9bd4-1c49012d0214 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:00.727555 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5af7d2-f838-4c85-9bd4-1c49012d0214 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:00.728960 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5af7d2-f838-4c85-9bd4-1c49012d0214 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:00.737855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-cc5af7d2-f838-4c85-9bd4-1c49012d0214 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:22:00.737855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:00.737855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:22:00.737855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:22:00.737855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:22:00.737855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:22:00.737855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:22:00.737855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:22:00.737855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:22:00.737855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:00.737855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:22:00.737855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:00.737855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:00.737855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:00.737855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:00.737855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:00.738807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:00.738807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:00.738807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:00.738807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:00.738807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:00.738807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:00.738807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:00.738807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:22:00.738807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:22:00.738807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:22:00.738807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:22:00.738807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:00.738807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:00.738807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:00.738807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:00.738807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:00.738807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:00.739658 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:00.739658 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:00.739658 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:00.739658 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:00.739658 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:00.739658 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:00.739658 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:00.739658 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:00.739658 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:00.739658 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:00.739658 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:22:00.739658 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:22:00.739658 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:22:00.739658 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:00.739658 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 457/907] 10.222.0.18 () {56 vars in 1132 bytes} [Thu Sep 1 16:22:00 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:22:00.751389 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111c709-4030-44e1-bcd8-2366ae1f2af1 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:00.767469 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:00.859187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111c709-4030-44e1-bcd8-2366ae1f2af1 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:00.871485 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5af7d2-f838-4c85-9bd4-1c49012d0214 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:00.898393 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c31b3-aff5-4081-b9d4-0aef25f59c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:00.899005 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c31b3-aff5-4081-b9d4-0aef25f59c6b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:00.899282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c31b3-aff5-4081-b9d4-0aef25f59c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:00.899587 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c31b3-aff5-4081-b9d4-0aef25f59c6b None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:00.900216 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c31b3-aff5-4081-b9d4-0aef25f59c6b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:00.900518 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c31b3-aff5-4081-b9d4-0aef25f59c6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:00.901867 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c31b3-aff5-4081-b9d4-0aef25f59c6b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:00.906922 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 451/908] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Sep 1 16:22:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:00.912305 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c31b3-aff5-4081-b9d4-0aef25f59c6b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:00.993047 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5af7d2-f838-4c85-9bd4-1c49012d0214 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:01.005563 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:01.033928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c31b3-aff5-4081-b9d4-0aef25f59c6b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:01.034328 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd2e566-a3e2-42c8-94c9-2172115d25d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:01.034944 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2e566-a3e2-42c8-94c9-2172115d25d1 None admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:01.035223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2e566-a3e2-42c8-94c9-2172115d25d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:01.035497 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2e566-a3e2-42c8-94c9-2172115d25d1 None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:01.036156 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd2e566-a3e2-42c8-94c9-2172115d25d1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:01.037733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd2e566-a3e2-42c8-94c9-2172115d25d1 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:01.045223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:01.059889 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 458/909] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Sep 1 16:22:00 2022] POST /identity/v3/services => generated 241 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:22:01.091153 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745cc73f-9abb-4551-9f80-dcbdd0965933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:01.091883 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745cc73f-9abb-4551-9f80-dcbdd0965933 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:01.092283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745cc73f-9abb-4551-9f80-dcbdd0965933 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:01.092627 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745cc73f-9abb-4551-9f80-dcbdd0965933 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:01.093407 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745cc73f-9abb-4551-9f80-dcbdd0965933 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:01.093827 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745cc73f-9abb-4551-9f80-dcbdd0965933 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:01.095359 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745cc73f-9abb-4551-9f80-dcbdd0965933 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:01.101853 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 452/910] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Sep 1 16:22:00 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:01.108304 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd2e566-a3e2-42c8-94c9-2172115d25d1 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:01.224408 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd2e566-a3e2-42c8-94c9-2172115d25d1 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:01.261305 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6382a4-0696-4bca-9469-96fc2df1572a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:01.262222 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6382a4-0696-4bca-9469-96fc2df1572a None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:01.262700 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6382a4-0696-4bca-9469-96fc2df1572a None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:01.263148 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6382a4-0696-4bca-9469-96fc2df1572a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:01.264012 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6382a4-0696-4bca-9469-96fc2df1572a None admin] RBAC: Adding query filter params (interface=public, service_id=074ea7bfdb754a979aaf8c6d4dc38eab, region_id=RegionOne) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:01.264500 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6382a4-0696-4bca-9469-96fc2df1572a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:01.266053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6382a4-0696-4bca-9469-96fc2df1572a None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:01.271604 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 459/911] 10.222.0.18 () {56 vars in 1251 bytes} [Thu Sep 1 16:22:01 2022] GET /identity/v3/endpoints?service_id=074ea7bfdb754a979aaf8c6d4dc38eab&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:01.276966 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745cc73f-9abb-4551-9f80-dcbdd0965933 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:01.385027 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745cc73f-9abb-4551-9f80-dcbdd0965933 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:01.421040 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7a946-927c-4a5b-b0b5-2406da0662e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:01.421656 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7a946-927c-4a5b-b0b5-2406da0662e4 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:01.421940 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7a946-927c-4a5b-b0b5-2406da0662e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:01.422221 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7a946-927c-4a5b-b0b5-2406da0662e4 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:01.422853 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a7a946-927c-4a5b-b0b5-2406da0662e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:01.424208 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a7a946-927c-4a5b-b0b5-2406da0662e4 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:01.426004 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:01.430848 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 453/912] 10.222.0.18 () {56 vars in 1087 bytes} [Thu Sep 1 16:22:01 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:22:02.460696 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3c6ea-0b46-493a-a832-f801431bc8a8 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:02.461246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3c6ea-0b46-493a-a832-f801431bc8a8 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:02.461246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3c6ea-0b46-493a-a832-f801431bc8a8 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:02.462927 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 460/913] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:02.468181 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bce279-b19f-4283-b255-8136d1485247 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:02.468589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bce279-b19f-4283-b255-8136d1485247 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:02.468726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bce279-b19f-4283-b255-8136d1485247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:02.471532 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962eb368-b230-42ff-934c-6233dff2c3ea None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:02.471817 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962eb368-b230-42ff-934c-6233dff2c3ea None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:02.472088 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962eb368-b230-42ff-934c-6233dff2c3ea None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:02.473324 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 461/914] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:02.479240 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b75497-88fd-4109-8762-3a99ded3a88c None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:02.479517 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b75497-88fd-4109-8762-3a99ded3a88c None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:02.479787 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b75497-88fd-4109-8762-3a99ded3a88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:02.495456 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-25bce279-b19f-4283-b255-8136d1485247 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:02.496700 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-25bce279-b19f-4283-b255-8136d1485247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:02.502913 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-27b75497-88fd-4109-8762-3a99ded3a88c None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:02.504260 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-27b75497-88fd-4109-8762-3a99ded3a88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:02.570212 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:02.586648 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 454/915] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:22:02.591106 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 462/916] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:22:02.851840 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bce279-b19f-4283-b255-8136d1485247 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:02.860830 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b75497-88fd-4109-8762-3a99ded3a88c None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:02.966194 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b75497-88fd-4109-8762-3a99ded3a88c None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:02.983217 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bce279-b19f-4283-b255-8136d1485247 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:03.002873 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112bc0e-be83-45fc-8aeb-13063954f9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:03.003511 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112bc0e-be83-45fc-8aeb-13063954f9d4 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:03.003796 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112bc0e-be83-45fc-8aeb-13063954f9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:03.004074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112bc0e-be83-45fc-8aeb-13063954f9d4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:03.004671 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112bc0e-be83-45fc-8aeb-13063954f9d4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:03.006055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112bc0e-be83-45fc-8aeb-13063954f9d4 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:03.015489 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-8112bc0e-be83-45fc-8aeb-13063954f9d4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:22:03.015489 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:03.015489 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:22:03.015489 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:22:03.015489 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:22:03.015489 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:22:03.015489 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:22:03.015489 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:22:03.015489 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:22:03.015489 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:03.015489 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:22:03.015489 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:03.015489 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:03.015489 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:03.015489 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:03.015489 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:03.015489 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:03.017802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:03.017802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:03.017802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:03.017802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:03.017802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:03.017802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:03.017802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:22:03.017802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:22:03.017802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:22:03.017802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:22:03.017802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:03.017802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:03.017802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:03.017802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:03.017802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:03.017802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:03.019792 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:03.019792 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:03.019792 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:03.019792 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:03.019792 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:03.019792 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:03.019792 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:03.019792 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:03.019792 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:03.019792 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:03.019792 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:22:03.019792 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:22:03.019792 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:22:03.019792 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:03.021609 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:03.021824 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 463/917] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 1 16:22:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:03.023182 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b0836-ede7-4875-a24f-636414f94581 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:03.023485 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b0836-ede7-4875-a24f-636414f94581 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:03.023745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b0836-ede7-4875-a24f-636414f94581 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:03.029455 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 464/918] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:03.029455 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b0836-ede7-4875-a24f-636414f94581 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:03.030127 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fad559-35da-4e6a-ac24-d38597d7d0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:03.030127 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fad559-35da-4e6a-ac24-d38597d7d0e9 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:03.030127 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fad559-35da-4e6a-ac24-d38597d7d0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:03.030127 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fad559-35da-4e6a-ac24-d38597d7d0e9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:03.039210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fad559-35da-4e6a-ac24-d38597d7d0e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:03.044678 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-78fad559-35da-4e6a-ac24-d38597d7d0e9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:22:03.044678 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:03.044678 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:03.044678 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:03.044678 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:03.044678 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:03.044678 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:03.044678 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:03.044678 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:03.044678 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:03.044678 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:03.044678 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:03.044678 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:03.044678 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:03.044678 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:03.044678 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:03.045777 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:03.045777 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:03.045777 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:03.045777 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:03.045777 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:03.045777 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:03.045777 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:03.045777 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:03.045777 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:03.045777 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:03.045777 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:03.045777 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:03.045777 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:03.045777 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:03.045777 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:03.045777 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:03.046874 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:03.046874 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:03.046874 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:03.046874 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:03.046874 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:03.046874 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:03.046874 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:03.046874 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:03.046874 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:03.046874 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:03.046874 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:22:03.046874 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:03.046874 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 455/919] 10.222.0.18 () {56 vars in 1120 bytes} [Thu Sep 1 16:22:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:03.048184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a480d-c085-4c4b-8d08-6279b7805c7c None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:03.048470 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a480d-c085-4c4b-8d08-6279b7805c7c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:03.048741 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a480d-c085-4c4b-8d08-6279b7805c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:03.066219 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-b67a480d-c085-4c4b-8d08-6279b7805c7c None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:03.067434 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b67a480d-c085-4c4b-8d08-6279b7805c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:03.091943 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:03.139419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b0836-ede7-4875-a24f-636414f94581 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:03.150129 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 456/920] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:22:03.152908 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a480d-c085-4c4b-8d08-6279b7805c7c None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:03.178680 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8d66c-8518-4fbc-aa2c-44d8887be59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:03.179595 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8d66c-8518-4fbc-aa2c-44d8887be59e None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:03.180081 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8d66c-8518-4fbc-aa2c-44d8887be59e None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:03.180547 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8d66c-8518-4fbc-aa2c-44d8887be59e None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:03.181522 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e8d66c-8518-4fbc-aa2c-44d8887be59e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:03.182019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e8d66c-8518-4fbc-aa2c-44d8887be59e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:03.183670 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e8d66c-8518-4fbc-aa2c-44d8887be59e None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:03.188813 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 465/921] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 1 16:22:03 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:03.194934 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8d66c-8518-4fbc-aa2c-44d8887be59e None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:03.273332 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a480d-c085-4c4b-8d08-6279b7805c7c None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:03.296775 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:03.314050 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b9e47-6d55-437c-b972-50271d68c80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:03.315034 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b9e47-6d55-437c-b972-50271d68c80d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:03.315523 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b9e47-6d55-437c-b972-50271d68c80d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:03.315811 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8d66c-8518-4fbc-aa2c-44d8887be59e None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:03.316233 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b9e47-6d55-437c-b972-50271d68c80d None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:03.317306 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b9e47-6d55-437c-b972-50271d68c80d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:03.317816 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b9e47-6d55-437c-b972-50271d68c80d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:03.319485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b9e47-6d55-437c-b972-50271d68c80d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:03.334378 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 457/922] 10.222.0.18 () {56 vars in 1115 bytes} [Thu Sep 1 16:22:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:03.335517 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b9e47-6d55-437c-b972-50271d68c80d None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:03.336513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:03.358637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbeaede-28c5-4592-9ba8-ff810eb21cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:03.359197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbeaede-28c5-4592-9ba8-ff810eb21cbb None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:03.359481 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbeaede-28c5-4592-9ba8-ff810eb21cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:03.359754 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbeaede-28c5-4592-9ba8-ff810eb21cbb None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:03.360384 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbeaede-28c5-4592-9ba8-ff810eb21cbb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:03.360715 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbeaede-28c5-4592-9ba8-ff810eb21cbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:03.362097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbeaede-28c5-4592-9ba8-ff810eb21cbb None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:03.374717 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 466/923] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Sep 1 16:22:03 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:03.376305 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c7726-37b7-4547-9eef-a01cc89d1223 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:03.376717 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c7726-37b7-4547-9eef-a01cc89d1223 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:03.377001 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c7726-37b7-4547-9eef-a01cc89d1223 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:03.378391 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 467/924] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:03.382869 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c7726-37b7-4547-9eef-a01cc89d1223 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:03.445505 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b9e47-6d55-437c-b972-50271d68c80d None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:03.478849 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c10a9a-6cd1-41f2-bbd6-fa0643f4f67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:03.479477 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c10a9a-6cd1-41f2-bbd6-fa0643f4f67d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:03.479759 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c10a9a-6cd1-41f2-bbd6-fa0643f4f67d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:03.480034 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c10a9a-6cd1-41f2-bbd6-fa0643f4f67d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:03.485250 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c7726-37b7-4547-9eef-a01cc89d1223 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:03.490999 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c10a9a-6cd1-41f2-bbd6-fa0643f4f67d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:03.492856 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c10a9a-6cd1-41f2-bbd6-fa0643f4f67d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:03.508616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-09c10a9a-6cd1-41f2-bbd6-fa0643f4f67d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:22:03.508616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:03.508616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:03.508616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:03.508616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:03.508616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:03.508616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:03.508616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:03.508616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:03.508616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:03.508616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:03.508616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:03.508616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:22:03.508616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:22:03.508616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:22:03.508616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:22:03.510037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:03.510037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:03.510037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:03.510037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:03.510037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:03.510037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:03.510037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:03.510037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:03.510037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:03.510037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:03.510037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:03.510037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:03.510037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:03.510037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:03.510037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:03.510037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:03.510037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:22:03.511380 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:22:03.511380 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:22:03.511380 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:03.511380 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 458/925] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:03.511610 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f1f37-1a9f-4d11-bddf-f00d93dc2a8c None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:03.511802 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f1f37-1a9f-4d11-bddf-f00d93dc2a8c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:03.512079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f1f37-1a9f-4d11-bddf-f00d93dc2a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:03.524365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cb7fd-1863-42eb-a1f3-54d7b0be0179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:03.525492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cb7fd-1863-42eb-a1f3-54d7b0be0179 None admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:03.525771 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cb7fd-1863-42eb-a1f3-54d7b0be0179 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:03.526043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cb7fd-1863-42eb-a1f3-54d7b0be0179 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:03.526739 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785cb7fd-1863-42eb-a1f3-54d7b0be0179 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:03.528128 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785cb7fd-1863-42eb-a1f3-54d7b0be0179 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:03.529667 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-2d5f1f37-1a9f-4d11-bddf-f00d93dc2a8c None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:03.530883 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2d5f1f37-1a9f-4d11-bddf-f00d93dc2a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:03.546847 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 468/926] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:22:03 2022] POST /identity/v3/endpoints => generated 346 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:22:03.547642 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cb7fd-1863-42eb-a1f3-54d7b0be0179 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:03.572454 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:03.594012 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:03.626500 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 459/927] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:22:03.627349 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f1f37-1a9f-4d11-bddf-f00d93dc2a8c None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:03.691859 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cb7fd-1863-42eb-a1f3-54d7b0be0179 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:03.732058 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad4298-b64f-4da3-9ad6-13dd16800c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:03.732922 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad4298-b64f-4da3-9ad6-13dd16800c90 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:03.733383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad4298-b64f-4da3-9ad6-13dd16800c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:03.733667 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad4298-b64f-4da3-9ad6-13dd16800c90 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:03.734305 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ad4298-b64f-4da3-9ad6-13dd16800c90 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:03.735692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ad4298-b64f-4da3-9ad6-13dd16800c90 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:03.746710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-30ad4298-b64f-4da3-9ad6-13dd16800c90 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:22:03.746710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:03.746710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:22:03.746710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:22:03.746710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:22:03.746710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:22:03.746710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:22:03.746710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:22:03.746710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:22:03.746710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:03.746710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:22:03.746710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:03.746710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:03.746710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:03.746710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:03.746710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:03.746710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:03.747681 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:03.747681 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:03.747681 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:03.747681 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:03.747681 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:03.747681 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:03.747681 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:22:03.747681 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:22:03.747681 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:22:03.747681 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:22:03.747681 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:03.747681 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:03.747681 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:03.747681 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:03.747681 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:03.747681 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:03.749392 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:03.749392 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:03.749392 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:03.749392 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:03.749392 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:03.749392 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:03.749392 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:03.749392 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:03.749392 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:03.749392 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:03.749392 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:22:03.749392 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:22:03.749392 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:22:03.749392 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:03.749392 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 469/928] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Sep 1 16:22:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:03.752784 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f1f37-1a9f-4d11-bddf-f00d93dc2a8c None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:03.753729 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad4298-b64f-4da3-9ad6-13dd16800c90 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:03.800172 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0baa62e-e833-4d49-9def-67e0fc78ef5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:03.800804 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0baa62e-e833-4d49-9def-67e0fc78ef5c None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:03.801149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0baa62e-e833-4d49-9def-67e0fc78ef5c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:03.801488 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0baa62e-e833-4d49-9def-67e0fc78ef5c None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:03.802194 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0baa62e-e833-4d49-9def-67e0fc78ef5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:03.802613 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0baa62e-e833-4d49-9def-67e0fc78ef5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:03.803987 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0baa62e-e833-4d49-9def-67e0fc78ef5c None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:03.821336 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 460/929] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:22:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:03.827176 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0baa62e-e833-4d49-9def-67e0fc78ef5c None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:03.849855 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:03.879261 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad4298-b64f-4da3-9ad6-13dd16800c90 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:03.880932 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:03.919874 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f917de9a-0d9c-4189-aa95-8ec732e3ac07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:03.920485 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917de9a-0d9c-4189-aa95-8ec732e3ac07 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:03.920763 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917de9a-0d9c-4189-aa95-8ec732e3ac07 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:03.921036 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917de9a-0d9c-4189-aa95-8ec732e3ac07 None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:03.921664 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f917de9a-0d9c-4189-aa95-8ec732e3ac07 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:03.921963 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f917de9a-0d9c-4189-aa95-8ec732e3ac07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:03.923359 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f917de9a-0d9c-4189-aa95-8ec732e3ac07 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:03.928480 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 470/930] 10.222.0.18 () {56 vars in 1122 bytes} [Thu Sep 1 16:22:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:03.929143 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f917de9a-0d9c-4189-aa95-8ec732e3ac07 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:03.953401 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0baa62e-e833-4d49-9def-67e0fc78ef5c None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:03.989921 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733ff2c9-0cb7-4a0d-acbb-1c9eb175b30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:03.990706 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ff2c9-0cb7-4a0d-acbb-1c9eb175b30f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:03.990987 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ff2c9-0cb7-4a0d-acbb-1c9eb175b30f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:03.991368 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ff2c9-0cb7-4a0d-acbb-1c9eb175b30f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:04.000523 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733ff2c9-0cb7-4a0d-acbb-1c9eb175b30f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:04.001954 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733ff2c9-0cb7-4a0d-acbb-1c9eb175b30f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:04.014371 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-733ff2c9-0cb7-4a0d-acbb-1c9eb175b30f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:22:04.014371 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:04.014371 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:04.014371 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:04.014371 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:04.014371 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:04.014371 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:04.014371 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:04.014371 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:04.014371 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:04.014371 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:04.014371 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:04.014371 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:22:04.014371 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:22:04.014371 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:22:04.014371 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:22:04.015300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:04.015300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:04.015300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:22:04.015300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:22:04.015300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:22:04.015300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:22:04.015300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:04.015300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:04.015300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:04.015300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:04.015300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:04.015300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:04.015300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:04.015300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:04.015300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:04.015300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:04.015300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:04.015300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:04.016235 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:04.016235 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:04.016235 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:22:04.016235 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:22:04.016235 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:22:04.016235 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:22:04.016235 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:22:04.016235 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:22:04.016235 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:22:04.016235 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:04.016235 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 461/931] 10.222.0.18 () {56 vars in 1117 bytes} [Thu Sep 1 16:22:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:22:04.016799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733ff2c9-0cb7-4a0d-acbb-1c9eb175b30f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:04.038842 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f917de9a-0d9c-4189-aa95-8ec732e3ac07 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:04.075276 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f005a32-53ab-48e3-abf3-aaada7bc2698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:04.075921 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f005a32-53ab-48e3-abf3-aaada7bc2698 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:04.076215 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f005a32-53ab-48e3-abf3-aaada7bc2698 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:04.076496 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f005a32-53ab-48e3-abf3-aaada7bc2698 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:04.077156 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f005a32-53ab-48e3-abf3-aaada7bc2698 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:04.078695 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f005a32-53ab-48e3-abf3-aaada7bc2698 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:04.087844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-2f005a32-53ab-48e3-abf3-aaada7bc2698 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:22:04.087844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:04.087844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:22:04.087844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:22:04.087844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:22:04.087844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:22:04.087844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:22:04.087844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:22:04.087844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:22:04.087844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:04.087844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:22:04.087844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:04.087844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:04.087844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:04.087844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:04.087844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:04.089019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:04.089019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:04.089019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:04.089019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:04.089019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:04.089019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:04.089019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:04.089019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:22:04.089019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:22:04.089019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:22:04.089019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:22:04.089019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:04.089019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:04.089019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:04.089019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:04.089019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:04.089019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:04.090036 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:04.090036 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:04.090036 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:04.090036 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:04.090036 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:04.090036 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:04.090036 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:04.090036 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:04.090036 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:04.090036 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:04.090036 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:22:04.090036 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:22:04.090036 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:22:04.090036 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:04.090036 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 471/932] 10.222.0.18 () {56 vars in 1132 bytes} [Thu Sep 1 16:22:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:04.090869 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f005a32-53ab-48e3-abf3-aaada7bc2698 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:04.107588 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:04.122362 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:04.132253 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733ff2c9-0cb7-4a0d-acbb-1c9eb175b30f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:04.169677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc643b-1942-490d-852b-0fed7902b030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:04.170322 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc643b-1942-490d-852b-0fed7902b030 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:04.170607 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc643b-1942-490d-852b-0fed7902b030 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:04.170881 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc643b-1942-490d-852b-0fed7902b030 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:04.171507 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc643b-1942-490d-852b-0fed7902b030 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:04.171811 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc643b-1942-490d-852b-0fed7902b030 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:04.173567 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc643b-1942-490d-852b-0fed7902b030 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:04.184669 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 462/933] 10.222.0.18 () {56 vars in 1122 bytes} [Thu Sep 1 16:22:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:04.185402 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc643b-1942-490d-852b-0fed7902b030 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:04.198277 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f005a32-53ab-48e3-abf3-aaada7bc2698 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:04.233490 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b50304-c491-48d8-9b07-ce699dbda9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:04.234102 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b50304-c491-48d8-9b07-ce699dbda9bc None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:04.234421 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b50304-c491-48d8-9b07-ce699dbda9bc None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:04.234695 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b50304-c491-48d8-9b07-ce699dbda9bc None admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:04.235418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b50304-c491-48d8-9b07-ce699dbda9bc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:04.235724 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b50304-c491-48d8-9b07-ce699dbda9bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:04.237146 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b50304-c491-48d8-9b07-ce699dbda9bc None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:04.255547 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 472/934] 10.222.0.18 () {56 vars in 1149 bytes} [Thu Sep 1 16:22:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:04.256200 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b50304-c491-48d8-9b07-ce699dbda9bc None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:04.296083 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc643b-1942-490d-852b-0fed7902b030 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:04.331763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e084b55-65e4-409c-b6e1-086bc1c2566d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:04.332373 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e084b55-65e4-409c-b6e1-086bc1c2566d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:04.332660 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e084b55-65e4-409c-b6e1-086bc1c2566d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:04.332936 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e084b55-65e4-409c-b6e1-086bc1c2566d None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:04.333570 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e084b55-65e4-409c-b6e1-086bc1c2566d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:04.333876 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e084b55-65e4-409c-b6e1-086bc1c2566d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:04.335260 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e084b55-65e4-409c-b6e1-086bc1c2566d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:04.340272 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 463/935] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Sep 1 16:22:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:04.341079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e084b55-65e4-409c-b6e1-086bc1c2566d None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:04.354697 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:04.357817 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:04.375935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b50304-c491-48d8-9b07-ce699dbda9bc None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:04.411160 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174d24a-07d8-4d76-807b-55f19ed08ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:04.411772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174d24a-07d8-4d76-807b-55f19ed08ecb None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:04.412121 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174d24a-07d8-4d76-807b-55f19ed08ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:04.412320 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174d24a-07d8-4d76-807b-55f19ed08ecb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:04.413029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7174d24a-07d8-4d76-807b-55f19ed08ecb None admin] RBAC: Adding query filter params (interface=public, service_id=6b871479336649389fd81413c43f513f, region_id=RegionOne) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:04.413341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7174d24a-07d8-4d76-807b-55f19ed08ecb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:04.415672 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7174d24a-07d8-4d76-807b-55f19ed08ecb None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:04.420405 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 473/936] 10.222.0.18 () {56 vars in 1251 bytes} [Thu Sep 1 16:22:04 2022] GET /identity/v3/endpoints?service_id=6b871479336649389fd81413c43f513f&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:04.421177 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174d24a-07d8-4d76-807b-55f19ed08ecb None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:04.451109 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e084b55-65e4-409c-b6e1-086bc1c2566d None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:04.486161 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52fbb9-47f3-4164-856e-4ea0f3db277a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:04.486819 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52fbb9-47f3-4164-856e-4ea0f3db277a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:04.487092 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52fbb9-47f3-4164-856e-4ea0f3db277a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:04.487364 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52fbb9-47f3-4164-856e-4ea0f3db277a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:04.499027 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c52fbb9-47f3-4164-856e-4ea0f3db277a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:04.500447 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c52fbb9-47f3-4164-856e-4ea0f3db277a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:04.515080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-1c52fbb9-47f3-4164-856e-4ea0f3db277a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:22:04.515080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:04.515080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:04.515080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:04.515080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:04.515080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:04.515080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:04.515080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:04.515080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:04.515080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:04.515080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:04.515080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:04.515080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:22:04.515080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:22:04.515080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:22:04.515080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:22:04.515984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:04.515984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:04.515984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:04.515984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:04.515984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:04.515984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:04.515984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:04.515984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:04.515984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:04.515984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:04.515984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:04.515984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:04.515984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:04.515984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:04.515984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:04.515984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:04.515984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:22:04.516846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:22:04.516846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:22:04.516846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:04.516846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 464/937] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:04.516846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52fbb9-47f3-4164-856e-4ea0f3db277a None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:04.532095 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174d24a-07d8-4d76-807b-55f19ed08ecb None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:04.568074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d457a7fd-4bce-46d2-a539-324ec9d8d7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:04.568692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457a7fd-4bce-46d2-a539-324ec9d8d7e9 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:04.568975 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457a7fd-4bce-46d2-a539-324ec9d8d7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:04.569251 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457a7fd-4bce-46d2-a539-324ec9d8d7e9 None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:04.569879 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d457a7fd-4bce-46d2-a539-324ec9d8d7e9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:04.570190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d457a7fd-4bce-46d2-a539-324ec9d8d7e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:04.571671 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d457a7fd-4bce-46d2-a539-324ec9d8d7e9 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:04.581321 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 474/938] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Sep 1 16:22:04 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:04.581321 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d457a7fd-4bce-46d2-a539-324ec9d8d7e9 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:04.586911 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:04.598698 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:04.636002 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52fbb9-47f3-4164-856e-4ea0f3db277a None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:04.673625 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f48af2-329f-4290-a961-64ff9495eddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:04.674365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f48af2-329f-4290-a961-64ff9495eddc None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:04.674651 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f48af2-329f-4290-a961-64ff9495eddc None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:04.674930 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f48af2-329f-4290-a961-64ff9495eddc None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:04.675646 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f48af2-329f-4290-a961-64ff9495eddc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:04.677127 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f48af2-329f-4290-a961-64ff9495eddc None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:04.694273 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 465/939] 10.222.0.18 () {56 vars in 1087 bytes} [Thu Sep 1 16:22:04 2022] GET /identity/v3/services => generated 1497 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:22:04.694507 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d457a7fd-4bce-46d2-a539-324ec9d8d7e9 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:04.694605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f48af2-329f-4290-a961-64ff9495eddc None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:04.737987 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd7503-5f56-4d24-950e-9365cdf021f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:04.738834 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd7503-5f56-4d24-950e-9365cdf021f7 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:04.739293 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd7503-5f56-4d24-950e-9365cdf021f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:04.739622 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd7503-5f56-4d24-950e-9365cdf021f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:04.740290 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd7503-5f56-4d24-950e-9365cdf021f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:04.740597 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd7503-5f56-4d24-950e-9365cdf021f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:04.742139 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd7503-5f56-4d24-950e-9365cdf021f7 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:04.761871 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 475/940] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:22:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:04.762945 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd7503-5f56-4d24-950e-9365cdf021f7 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:04.835485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f48af2-329f-4290-a961-64ff9495eddc None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:04.856714 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:04.868938 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:04.884914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd7d54-068c-40e5-b31a-ba0c80d240b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:04.885531 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd7d54-068c-40e5-b31a-ba0c80d240b1 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:04.885810 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd7d54-068c-40e5-b31a-ba0c80d240b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:04.886082 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd7d54-068c-40e5-b31a-ba0c80d240b1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:04.886304 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd7503-5f56-4d24-950e-9365cdf021f7 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:04.886805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd7d54-068c-40e5-b31a-ba0c80d240b1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:04.888342 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd7d54-068c-40e5-b31a-ba0c80d240b1 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:04.918324 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 466/941] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:22:04 2022] POST /identity/v3/endpoints => generated 347 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:22:04.940310 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d1927-117e-4459-a81e-c49908775cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:04.941020 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d1927-117e-4459-a81e-c49908775cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:04.941490 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d1927-117e-4459-a81e-c49908775cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:04.941836 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d1927-117e-4459-a81e-c49908775cc0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:04.951069 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d1927-117e-4459-a81e-c49908775cc0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:04.952783 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d1927-117e-4459-a81e-c49908775cc0 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:04.966739 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-051d1927-117e-4459-a81e-c49908775cc0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:22:04.966739 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:04.966739 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:04.966739 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:04.966739 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:04.966739 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:04.966739 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:04.966739 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:04.966739 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:04.966739 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:04.966739 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:04.966739 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:04.966739 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:22:04.966739 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:22:04.966739 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:22:04.967703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:22:04.967703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:04.967703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:04.967703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:04.967703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:04.967703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:04.967703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:04.967703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:04.967703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:04.967703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:04.967703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:04.967703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:04.967703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:04.967703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:04.967703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:04.967703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:04.967703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:04.968586 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:22:04.968586 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:22:04.968586 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:22:04.968586 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:22:04.968586 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:22:04.968586 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:22:04.968586 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:22:04.968586 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:04.968586 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 476/942] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:22:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:04.974576 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd7d54-068c-40e5-b31a-ba0c80d240b1 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:05.092335 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd7d54-068c-40e5-b31a-ba0c80d240b1 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:05.129691 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e2d68-e2c8-44be-a385-d0723b3d16ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:05.130330 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e2d68-e2c8-44be-a385-d0723b3d16ae None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:05.130612 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e2d68-e2c8-44be-a385-d0723b3d16ae None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:05.130886 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e2d68-e2c8-44be-a385-d0723b3d16ae None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:05.131583 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248e2d68-e2c8-44be-a385-d0723b3d16ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:05.131888 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248e2d68-e2c8-44be-a385-d0723b3d16ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:05.133300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248e2d68-e2c8-44be-a385-d0723b3d16ae None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:05.147031 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 467/943] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:22:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:05.153329 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d1927-117e-4459-a81e-c49908775cc0 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:05.261536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d1927-117e-4459-a81e-c49908775cc0 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:05.292774 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:05.298259 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d312d-a3c4-4974-b0bb-b1a146357af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:05.299000 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d312d-a3c4-4974-b0bb-b1a146357af4 None admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:05.299286 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d312d-a3c4-4974-b0bb-b1a146357af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:05.299594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d312d-a3c4-4974-b0bb-b1a146357af4 None admin] PATH_INFO: `/v3/projects/36ce80c4e0d64562a9ab3961be76fad9/users/a5b3e53fd1314e17ad28c42a44e8eff3/roles/a889b69245b44da59d9020a0d39751cb` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:05.302143 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d312d-a3c4-4974-b0bb-b1a146357af4 None admin] RBAC: Authorizing `identity:create_grant(project_id=36ce80c4e0d64562a9ab3961be76fad9, user_id=a5b3e53fd1314e17ad28c42a44e8eff3, role_id=a889b69245b44da59d9020a0d39751cb)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:05.303681 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d312d-a3c4-4974-b0bb-b1a146357af4 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:05.320339 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 477/944] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 1 16:22:05 2022] PUT /identity/v3/projects/36ce80c4e0d64562a9ab3961be76fad9/users/a5b3e53fd1314e17ad28c42a44e8eff3/roles/a889b69245b44da59d9020a0d39751cb => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:22:05.565184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e43ab-1860-475c-9589-fb97ae66f960 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:05.565702 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e43ab-1860-475c-9589-fb97ae66f960 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:05.565702 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e43ab-1860-475c-9589-fb97ae66f960 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:05.567340 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 468/945] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:05.572509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac3eec-a919-4c84-bfb6-c3fe09313f52 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:05.572923 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac3eec-a919-4c84-bfb6-c3fe09313f52 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:05.573097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac3eec-a919-4c84-bfb6-c3fe09313f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:05.592303 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-a3ac3eec-a919-4c84-bfb6-c3fe09313f52 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:05.593510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-a3ac3eec-a919-4c84-bfb6-c3fe09313f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:05.665052 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 478/946] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:22:05.934990 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e43ab-1860-475c-9589-fb97ae66f960 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:05.983310 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:06.044953 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e43ab-1860-475c-9589-fb97ae66f960 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:06.080226 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0be711-a5b9-4b3d-9349-3acab79fa689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:06.080844 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0be711-a5b9-4b3d-9349-3acab79fa689 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:06.081131 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0be711-a5b9-4b3d-9349-3acab79fa689 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:06.081403 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0be711-a5b9-4b3d-9349-3acab79fa689 None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:06.082054 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0be711-a5b9-4b3d-9349-3acab79fa689 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:06.083522 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0be711-a5b9-4b3d-9349-3acab79fa689 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:06.086166 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: WARNING keystone.server.flask.common [None req-9a0be711-a5b9-4b3d-9349-3acab79fa689 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 01 16:22:06.109679 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.sql.core [None req-9a0be711-a5b9-4b3d-9349-3acab79fa689 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 16:22:06.109679 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 16:22:06.109679 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [parameters: {'id': '599fc753eac54f7199879d71993b4a6f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 16:22:06.109679 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81551) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 16:22:06.117460 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-9a0be711-a5b9-4b3d-9349-3acab79fa689 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 599fc753eac54f7199879d71993b4a6f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 599fc753eac54f7199879d71993b4a6f. Sep 01 16:22:06.117460 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:06.117460 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 16:22:06.117460 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 16:22:06.117460 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 16:22:06.117460 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 16:22:06.117460 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 16:22:06.117460 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 16:22:06.117460 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 16:22:06.117460 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application conn.query(q) Sep 01 16:22:06.117460 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 16:22:06.117460 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 16:22:06.117460 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 16:22:06.118799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application result.read() Sep 01 16:22:06.118799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 16:22:06.118799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 16:22:06.118799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 16:22:06.118799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 16:22:06.118799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 16:22:06.118799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 16:22:06.118799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 16:22:06.118799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 16:22:06.118799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 16:22:06.118799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:06.118799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 16:22:06.118799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:06.118799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:06.118799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 16:22:06.118799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 16:22:06.118799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 01 16:22:06.118799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 01 16:22:06.120791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 16:22:06.120791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application next(self.gen) Sep 01 16:22:06.120791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 16:22:06.120791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application yield resource Sep 01 16:22:06.120791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 16:22:06.120791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application next(self.gen) Sep 01 16:22:06.120791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 16:22:06.120791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 16:22:06.120791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 16:22:06.120791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 16:22:06.120791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 16:22:06.120791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise self.value Sep 01 16:22:06.120791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 16:22:06.120791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 16:22:06.120791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 16:22:06.120791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application session.commit() Sep 01 16:22:06.120791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 16:22:06.122468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 16:22:06.122468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 16:22:06.122468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 16:22:06.122468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 16:22:06.122468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self.session.flush() Sep 01 16:22:06.122468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 16:22:06.122468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 16:22:06.122468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 16:22:06.122468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 16:22:06.122468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 16:22:06.122468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application compat.raise_( Sep 01 16:22:06.122468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 16:22:06.122468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception Sep 01 16:22:06.122468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 16:22:06.122468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 16:22:06.122468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 16:22:06.122468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 16:22:06.123668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 16:22:06.123668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 16:22:06.123668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 16:22:06.123668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 16:22:06.123668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 16:22:06.123668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 16:22:06.123668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 16:22:06.123668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 16:22:06.123668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 16:22:06.123668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 16:22:06.123668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 16:22:06.123668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 16:22:06.123668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 16:22:06.123668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 16:22:06.123668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 16:22:06.124668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 16:22:06.124668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 16:22:06.124668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception Sep 01 16:22:06.124668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 16:22:06.124668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 16:22:06.124668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 16:22:06.124668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 16:22:06.124668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 16:22:06.124668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 16:22:06.124668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 16:22:06.124668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application conn.query(q) Sep 01 16:22:06.124668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 16:22:06.124668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 16:22:06.124668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 16:22:06.124668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application result.read() Sep 01 16:22:06.124668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 16:22:06.124668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 16:22:06.125726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 16:22:06.125726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 16:22:06.125726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 16:22:06.125726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 16:22:06.125726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 16:22:06.125726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 16:22:06.125726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 16:22:06.125726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 16:22:06.125726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [parameters: {'id': '599fc753eac54f7199879d71993b4a6f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 16:22:06.125726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 16:22:06.125726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:06.125726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:22:06.125726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:06.125726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:06.125726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 01 16:22:06.128110 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 01 16:22:06.128110 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 16:22:06.128110 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 16:22:06.128110 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 01 16:22:06.128110 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:06.128110 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:22:06.128110 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:06.128110 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:06.128110 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:06.128110 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:06.128110 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:06.128110 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:06.128110 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:06.128110 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:06.128110 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:06.128110 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:06.128110 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:06.134385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:06.134385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 01 16:22:06.134385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 01 16:22:06.134385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:06.134385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:06.134385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 01 16:22:06.134385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 01 16:22:06.134385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 599fc753eac54f7199879d71993b4a6f. Sep 01 16:22:06.134385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:06.134385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 469/947] 10.222.0.18 () {60 vars in 1140 bytes} [Thu Sep 1 16:22:05 2022] POST /identity/v3/projects => generated 262 bytes in 185 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:06.137475 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac3eec-a919-4c84-bfb6-c3fe09313f52 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:06.239510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac3eec-a919-4c84-bfb6-c3fe09313f52 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:06.263790 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:06.276666 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67097c8a-2c21-49cc-8276-ab1969839969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:06.277362 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67097c8a-2c21-49cc-8276-ab1969839969 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:06.277649 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67097c8a-2c21-49cc-8276-ab1969839969 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:06.277947 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67097c8a-2c21-49cc-8276-ab1969839969 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:06.284913 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67097c8a-2c21-49cc-8276-ab1969839969 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:06.286395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67097c8a-2c21-49cc-8276-ab1969839969 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:06.298183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-67097c8a-2c21-49cc-8276-ab1969839969 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:22:06.298183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:06.298183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:06.298183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:06.298183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:06.298183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:06.298183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:06.298183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:06.298183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:06.298183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:06.298183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:06.298183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:06.298183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:22:06.298183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:22:06.298183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:22:06.299097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:22:06.299097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:06.299097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:06.299097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:06.299097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:06.299097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:06.299097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:06.299097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:06.299097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:06.299097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:06.299097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:06.299097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:06.299097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:06.299097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:06.299097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:06.299097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:06.299097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:06.299974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:22:06.299974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:22:06.299974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:22:06.299974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:22:06.299974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:22:06.299974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:22:06.299974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:22:06.299974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:06.299974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 479/948] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:22:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:06.306116 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0be711-a5b9-4b3d-9349-3acab79fa689 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:06.412470 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:06.419307 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0be711-a5b9-4b3d-9349-3acab79fa689 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:06.455975 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31299e-60d1-4c41-bdaa-13d65401baf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:06.456618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31299e-60d1-4c41-bdaa-13d65401baf5 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:06.456904 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31299e-60d1-4c41-bdaa-13d65401baf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:06.457186 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31299e-60d1-4c41-bdaa-13d65401baf5 None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:06.457880 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c31299e-60d1-4c41-bdaa-13d65401baf5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:06.458188 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c31299e-60d1-4c41-bdaa-13d65401baf5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:06.459935 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c31299e-60d1-4c41-bdaa-13d65401baf5 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:06.482198 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 470/949] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Sep 1 16:22:06 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:06.677265 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7669a16-ea64-4d6e-9917-a61abfcee545 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:06.677908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7669a16-ea64-4d6e-9917-a61abfcee545 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:06.677908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7669a16-ea64-4d6e-9917-a61abfcee545 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:06.679414 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 480/950] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:06.685409 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276f499-ac06-44ab-9209-15c22e6e8b9d None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:06.685907 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276f499-ac06-44ab-9209-15c22e6e8b9d None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:06.686060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276f499-ac06-44ab-9209-15c22e6e8b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:06.706602 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-1276f499-ac06-44ab-9209-15c22e6e8b9d None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:06.708144 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-1276f499-ac06-44ab-9209-15c22e6e8b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:06.779936 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 471/951] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:22:06.970730 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e1a92-5a38-466a-876f-86b1d1f6a05e None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:06.971362 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e1a92-5a38-466a-876f-86b1d1f6a05e None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:06.971854 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e1a92-5a38-466a-876f-86b1d1f6a05e None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:06.973660 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 481/952] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:06.979703 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3ca4c-b471-4148-881b-5e85c5b70c90 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:06.980254 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3ca4c-b471-4148-881b-5e85c5b70c90 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:06.980736 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3ca4c-b471-4148-881b-5e85c5b70c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:06.999840 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-53d3ca4c-b471-4148-881b-5e85c5b70c90 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:07.001365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-53d3ca4c-b471-4148-881b-5e85c5b70c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:07.047123 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e1a92-5a38-466a-876f-86b1d1f6a05e None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:07.074928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 472/953] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:22:07.157166 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e1a92-5a38-466a-876f-86b1d1f6a05e None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:07.191750 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a83d4-0e8c-4cfe-a011-0acf98197c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:07.192388 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a83d4-0e8c-4cfe-a011-0acf98197c49 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:07.192672 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a83d4-0e8c-4cfe-a011-0acf98197c49 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:07.192949 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a83d4-0e8c-4cfe-a011-0acf98197c49 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:07.193539 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a83d4-0e8c-4cfe-a011-0acf98197c49 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:07.194994 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a83d4-0e8c-4cfe-a011-0acf98197c49 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:07.204571 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-498a83d4-0e8c-4cfe-a011-0acf98197c49 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:22:07.204571 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:07.204571 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:22:07.204571 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:22:07.204571 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:22:07.204571 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:22:07.204571 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:22:07.204571 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:22:07.204571 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:22:07.204571 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:07.204571 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:22:07.204571 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:07.204571 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:07.204571 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:07.204571 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:07.204571 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:07.204571 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:07.205540 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:07.205540 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:07.205540 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:07.205540 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:07.205540 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:07.205540 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:07.205540 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:22:07.205540 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:22:07.205540 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:22:07.205540 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:22:07.205540 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:07.205540 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:07.205540 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:07.205540 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:07.205540 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:07.205540 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:07.206609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:07.206609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:07.206609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:07.206609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:07.206609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:07.206609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:07.206609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:07.206609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:07.206609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:07.206609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:07.206609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:22:07.206609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:22:07.206609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:22:07.206609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:07.206609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 482/954] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Sep 1 16:22:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:22:07.215709 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d3ca4c-b471-4148-881b-5e85c5b70c90 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:07.234700 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:07.318071 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982e531-46be-4996-8ec8-4edaacf7a0aa None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:07.318430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982e531-46be-4996-8ec8-4edaacf7a0aa None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:07.318665 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982e531-46be-4996-8ec8-4edaacf7a0aa None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:07.318854 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d3ca4c-b471-4148-881b-5e85c5b70c90 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:07.320250 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 483/955] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:07.330776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45df81-38d0-4bee-adfd-9d6ef3e3a87e None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:07.331593 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45df81-38d0-4bee-adfd-9d6ef3e3a87e None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:07.333419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45df81-38d0-4bee-adfd-9d6ef3e3a87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:07.362956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-8b45df81-38d0-4bee-adfd-9d6ef3e3a87e None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:07.363293 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74785edb-1948-4c2c-9b71-66549555d656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:07.363834 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74785edb-1948-4c2c-9b71-66549555d656 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:07.364118 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74785edb-1948-4c2c-9b71-66549555d656 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:07.364393 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74785edb-1948-4c2c-9b71-66549555d656 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:07.365028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74785edb-1948-4c2c-9b71-66549555d656 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:07.365337 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74785edb-1948-4c2c-9b71-66549555d656 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:07.365861 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-8b45df81-38d0-4bee-adfd-9d6ef3e3a87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:07.366737 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74785edb-1948-4c2c-9b71-66549555d656 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:07.372185 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 473/956] 10.222.0.18 () {56 vars in 1122 bytes} [Thu Sep 1 16:22:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:07.372970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74785edb-1948-4c2c-9b71-66549555d656 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:07.418796 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:07.450436 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 484/957] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:22:07.450436 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45df81-38d0-4bee-adfd-9d6ef3e3a87e None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:07.463411 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:07.486517 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74785edb-1948-4c2c-9b71-66549555d656 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:07.524840 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727fbeb1-851f-40f8-8acc-9700d054a088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:07.525475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727fbeb1-851f-40f8-8acc-9700d054a088 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:07.525800 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727fbeb1-851f-40f8-8acc-9700d054a088 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:07.526099 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727fbeb1-851f-40f8-8acc-9700d054a088 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:07.526696 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727fbeb1-851f-40f8-8acc-9700d054a088 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:07.528181 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727fbeb1-851f-40f8-8acc-9700d054a088 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:07.537844 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-727fbeb1-851f-40f8-8acc-9700d054a088 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:22:07.537844 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:07.537844 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:22:07.537844 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:22:07.537844 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:22:07.537844 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:22:07.537844 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:22:07.537844 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:22:07.537844 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:22:07.537844 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:07.537844 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:22:07.537844 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:07.537844 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:07.537844 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:07.537844 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:07.537844 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:07.537844 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:07.538978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:07.538978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:07.538978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:07.538978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:07.538978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:07.538978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:07.538978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:22:07.538978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:22:07.538978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:22:07.538978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:22:07.538978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:07.538978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:07.538978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:07.538978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:07.538978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:07.538978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:07.544978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:07.544978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:07.544978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:07.544978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:07.544978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:07.544978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:07.544978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:07.544978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:07.544978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:07.544978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:07.544978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:22:07.544978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:22:07.544978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:22:07.544978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:07.544978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 474/958] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:22:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:07.545823 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727fbeb1-851f-40f8-8acc-9700d054a088 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:07.587797 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45df81-38d0-4bee-adfd-9d6ef3e3a87e None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:07.637095 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794fe101-1a35-4d93-8cea-de03685a9adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:07.637860 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794fe101-1a35-4d93-8cea-de03685a9adc None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:07.638268 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794fe101-1a35-4d93-8cea-de03685a9adc None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:07.638553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794fe101-1a35-4d93-8cea-de03685a9adc None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:07.639186 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794fe101-1a35-4d93-8cea-de03685a9adc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:07.639490 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794fe101-1a35-4d93-8cea-de03685a9adc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:07.641670 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794fe101-1a35-4d93-8cea-de03685a9adc None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:07.648521 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 485/959] 10.222.0.18 () {56 vars in 1122 bytes} [Thu Sep 1 16:22:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:07.653919 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794fe101-1a35-4d93-8cea-de03685a9adc None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:07.665487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727fbeb1-851f-40f8-8acc-9700d054a088 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:07.701849 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91a92e-eabb-4cf9-bd3e-26309029cab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:07.702547 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91a92e-eabb-4cf9-bd3e-26309029cab2 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:07.702933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91a92e-eabb-4cf9-bd3e-26309029cab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:07.703207 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91a92e-eabb-4cf9-bd3e-26309029cab2 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:07.703831 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed91a92e-eabb-4cf9-bd3e-26309029cab2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:07.704132 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed91a92e-eabb-4cf9-bd3e-26309029cab2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:07.705474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed91a92e-eabb-4cf9-bd3e-26309029cab2 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:07.710741 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 475/960] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Sep 1 16:22:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:07.712068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:07.722348 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91a92e-eabb-4cf9-bd3e-26309029cab2 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:07.739393 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:07.778970 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794fe101-1a35-4d93-8cea-de03685a9adc None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:07.812587 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b317a-0167-413a-b997-77c8faad9a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:07.813189 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b317a-0167-413a-b997-77c8faad9a15 None admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:07.813468 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b317a-0167-413a-b997-77c8faad9a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:07.813741 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b317a-0167-413a-b997-77c8faad9a15 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:07.814430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b317a-0167-413a-b997-77c8faad9a15 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:07.815795 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b317a-0167-413a-b997-77c8faad9a15 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:07.836910 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 486/961] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:22:07 2022] POST /identity/v3/endpoints => generated 349 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:22:07.837646 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b317a-0167-413a-b997-77c8faad9a15 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:07.848804 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91a92e-eabb-4cf9-bd3e-26309029cab2 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:07.897414 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13995794-09c1-4f61-b918-561a5568501b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:07.898038 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13995794-09c1-4f61-b918-561a5568501b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:07.898351 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13995794-09c1-4f61-b918-561a5568501b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:07.898674 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13995794-09c1-4f61-b918-561a5568501b None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:07.899596 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13995794-09c1-4f61-b918-561a5568501b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:07.899792 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13995794-09c1-4f61-b918-561a5568501b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:07.901154 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13995794-09c1-4f61-b918-561a5568501b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:07.906880 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 476/962] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Sep 1 16:22:07 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:07.966813 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:07.974788 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b317a-0167-413a-b997-77c8faad9a15 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:08.017549 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5428d7-0115-4182-bdf5-b101e3168cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:08.018533 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5428d7-0115-4182-bdf5-b101e3168cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:08.018998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5428d7-0115-4182-bdf5-b101e3168cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:08.019423 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5428d7-0115-4182-bdf5-b101e3168cd1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:08.032272 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5428d7-0115-4182-bdf5-b101e3168cd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:08.037466 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-ea5428d7-0115-4182-bdf5-b101e3168cd1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:22:08.037466 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:08.037466 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:08.037466 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:08.037466 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:08.037466 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:08.037466 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:08.037466 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:08.037466 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:08.037466 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:08.037466 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:08.037466 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:08.037466 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:08.037466 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:08.037466 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:08.037466 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:08.038591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:08.038591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:08.038591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:08.038591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:08.038591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:08.038591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:08.038591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:08.038591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:08.038591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:08.038591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:08.038591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:08.038591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:08.038591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:08.038591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:08.038591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:08.038591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:08.039413 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:08.039413 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:08.039413 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:08.039413 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:08.039413 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:08.039413 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:08.039413 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:08.039413 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:08.039413 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:08.039413 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:08.039413 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:22:08.039413 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:08.040640 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 487/963] 10.222.0.18 () {56 vars in 1120 bytes} [Thu Sep 1 16:22:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:08.047548 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13995794-09c1-4f61-b918-561a5568501b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:08.120135 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:08.159661 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13995794-09c1-4f61-b918-561a5568501b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:08.196927 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6578ce3-9602-447f-b9f3-1939f6f038f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:08.197909 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6578ce3-9602-447f-b9f3-1939f6f038f7 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:08.198450 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6578ce3-9602-447f-b9f3-1939f6f038f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:08.198946 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6578ce3-9602-447f-b9f3-1939f6f038f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:08.199816 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6578ce3-9602-447f-b9f3-1939f6f038f7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:08.200357 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6578ce3-9602-447f-b9f3-1939f6f038f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:08.202006 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6578ce3-9602-447f-b9f3-1939f6f038f7 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:08.211309 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 477/964] 10.222.0.18 () {56 vars in 1115 bytes} [Thu Sep 1 16:22:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:08.219062 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5428d7-0115-4182-bdf5-b101e3168cd1 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:08.325466 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5428d7-0115-4182-bdf5-b101e3168cd1 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:08.362378 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a90609-0e88-4fb6-9268-284e89bb4087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:08.363459 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a90609-0e88-4fb6-9268-284e89bb4087 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:08.363951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a90609-0e88-4fb6-9268-284e89bb4087 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:08.364452 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a90609-0e88-4fb6-9268-284e89bb4087 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:08.375835 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a90609-0e88-4fb6-9268-284e89bb4087 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:08.377309 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a90609-0e88-4fb6-9268-284e89bb4087 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:08.392529 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-76a90609-0e88-4fb6-9268-284e89bb4087 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:22:08.392529 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:08.392529 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:08.392529 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:08.392529 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:08.392529 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:08.392529 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:08.392529 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:08.392529 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:08.392529 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:08.392529 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:08.392529 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:08.392529 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:22:08.392529 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:22:08.392529 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:22:08.392529 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:22:08.393526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:08.393526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:08.393526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:08.393526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:08.393526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:08.393526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:08.393526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:08.393526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:08.393526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:08.393526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:08.393526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:08.393526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:08.393526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:08.393526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:08.393526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:08.393526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:08.393526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:22:08.394452 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:22:08.394452 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:22:08.394452 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:08.394452 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 488/965] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:08.400332 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6578ce3-9602-447f-b9f3-1939f6f038f7 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:08.506657 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6578ce3-9602-447f-b9f3-1939f6f038f7 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:08.536710 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:08.544600 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c813afc-2aa3-465b-bcef-b34eaa13c85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:08.545284 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c813afc-2aa3-465b-bcef-b34eaa13c85f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:08.545572 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c813afc-2aa3-465b-bcef-b34eaa13c85f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:08.545847 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c813afc-2aa3-465b-bcef-b34eaa13c85f None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:08.546574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c813afc-2aa3-465b-bcef-b34eaa13c85f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:08.546883 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c813afc-2aa3-465b-bcef-b34eaa13c85f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:08.548241 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c813afc-2aa3-465b-bcef-b34eaa13c85f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:08.556735 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5115c9-ba53-4d23-bfd1-977db467285b None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:08.557025 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5115c9-ba53-4d23-bfd1-977db467285b None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:08.557301 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5115c9-ba53-4d23-bfd1-977db467285b None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:08.559779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 489/966] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:22:08.561108 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 478/967] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:22:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:08.566905 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c813afc-2aa3-465b-bcef-b34eaa13c85f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:08.567309 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd693c2-6f1b-401c-b80c-ead25b85b3bc None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:08.567505 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd693c2-6f1b-401c-b80c-ead25b85b3bc None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:08.567904 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd693c2-6f1b-401c-b80c-ead25b85b3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:08.591763 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-9dd693c2-6f1b-401c-b80c-ead25b85b3bc None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:08.593545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-9dd693c2-6f1b-401c-b80c-ead25b85b3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:08.597851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:08.676236 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c813afc-2aa3-465b-bcef-b34eaa13c85f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:08.686508 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 490/968] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:22:08.721284 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e9c37-05f2-4973-9875-bfd950c80f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:08.722580 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e9c37-05f2-4973-9875-bfd950c80f94 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:08.723063 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e9c37-05f2-4973-9875-bfd950c80f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:08.723523 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e9c37-05f2-4973-9875-bfd950c80f94 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:08.731232 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e9c37-05f2-4973-9875-bfd950c80f94 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:08.732877 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e9c37-05f2-4973-9875-bfd950c80f94 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:08.745257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-fb0e9c37-05f2-4973-9875-bfd950c80f94 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:22:08.745257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:08.745257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:08.745257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:08.745257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:08.745257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:08.745257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:08.745257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:08.745257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:08.745257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:08.745257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:08.745257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:08.745257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:22:08.745257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:22:08.745257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:22:08.745257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:22:08.746205 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:08.746205 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:08.746205 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:22:08.746205 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:22:08.746205 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:22:08.746205 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:22:08.746205 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:08.746205 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:08.746205 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:08.746205 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:08.746205 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:08.746205 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:08.746205 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:08.746205 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:08.746205 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:08.746205 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:08.746205 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:08.746205 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:08.747187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:08.747187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:08.747187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:22:08.747187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:22:08.747187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:22:08.747187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:22:08.747187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:22:08.747187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:22:08.747187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:22:08.747187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:08.749342 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 479/969] 10.222.0.18 () {56 vars in 1117 bytes} [Thu Sep 1 16:22:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:08.755066 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd693c2-6f1b-401c-b80c-ead25b85b3bc None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:08.856307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd693c2-6f1b-401c-b80c-ead25b85b3bc None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:08.891805 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ecef38-ac2c-4d47-a393-a768a0c6de47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:08.892422 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ecef38-ac2c-4d47-a393-a768a0c6de47 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:08.892703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ecef38-ac2c-4d47-a393-a768a0c6de47 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:08.892978 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ecef38-ac2c-4d47-a393-a768a0c6de47 None admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:08.893716 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ecef38-ac2c-4d47-a393-a768a0c6de47 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:08.894021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ecef38-ac2c-4d47-a393-a768a0c6de47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:08.895508 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ecef38-ac2c-4d47-a393-a768a0c6de47 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:08.904342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:08.913980 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 491/970] 10.222.0.18 () {56 vars in 1149 bytes} [Thu Sep 1 16:22:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:08.921953 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e9c37-05f2-4973-9875-bfd950c80f94 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:08.968164 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ecef38-ac2c-4d47-a393-a768a0c6de47 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:08.978505 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:09.034800 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e9c37-05f2-4973-9875-bfd950c80f94 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:09.073041 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bed76c-c265-481b-b9b8-b2659ee24b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:09.073659 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bed76c-c265-481b-b9b8-b2659ee24b06 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:09.073943 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bed76c-c265-481b-b9b8-b2659ee24b06 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:09.074220 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bed76c-c265-481b-b9b8-b2659ee24b06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:09.107546 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ecef38-ac2c-4d47-a393-a768a0c6de47 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:09.150547 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bed76c-c265-481b-b9b8-b2659ee24b06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:09.151043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ecfe45-a8fa-4886-a901-76860621e5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:09.151727 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ecfe45-a8fa-4886-a901-76860621e5fc None admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:09.152105 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ecfe45-a8fa-4886-a901-76860621e5fc None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:09.152469 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ecfe45-a8fa-4886-a901-76860621e5fc None admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:09.153172 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bed76c-c265-481b-b9b8-b2659ee24b06 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:09.153288 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ecfe45-a8fa-4886-a901-76860621e5fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:09.154783 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ecfe45-a8fa-4886-a901-76860621e5fc None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:09.158674 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: WARNING keystone.server.flask.common [None req-80ecfe45-a8fa-4886-a901-76860621e5fc None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 01 16:22:09.176225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.sql.core [None req-80ecfe45-a8fa-4886-a901-76860621e5fc None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 16:22:09.176225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 16:22:09.176225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [parameters: {'user_id': 'ff7f20fbe3aa498487f3795903840bf5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 16:22:09.176225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81552) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 16:22:09.199829 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-80ecfe45-a8fa-4886-a901-76860621e5fc None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 16:22:09.199829 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:09.199829 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 16:22:09.199829 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 16:22:09.199829 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 16:22:09.199829 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 16:22:09.199829 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 16:22:09.199829 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 16:22:09.199829 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 16:22:09.199829 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application conn.query(q) Sep 01 16:22:09.199829 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 16:22:09.199829 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 16:22:09.199829 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 16:22:09.199829 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application result.read() Sep 01 16:22:09.199829 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 16:22:09.200776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 16:22:09.200776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 16:22:09.200776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 16:22:09.200776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 16:22:09.200776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 16:22:09.200776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 16:22:09.200776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 16:22:09.200776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 16:22:09.200776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:09.200776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 16:22:09.200776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:09.200776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:09.200776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 16:22:09.200776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 16:22:09.200776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 01 16:22:09.200776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 01 16:22:09.200776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 16:22:09.200776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application next(self.gen) Sep 01 16:22:09.206894 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 16:22:09.206894 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application yield resource Sep 01 16:22:09.206894 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 16:22:09.206894 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application next(self.gen) Sep 01 16:22:09.206894 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 16:22:09.206894 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 16:22:09.206894 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 16:22:09.206894 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 16:22:09.206894 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 16:22:09.206894 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise self.value Sep 01 16:22:09.206894 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 16:22:09.206894 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 16:22:09.206894 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 16:22:09.206894 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application session.commit() Sep 01 16:22:09.206894 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 16:22:09.206894 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 16:22:09.206894 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 16:22:09.207812 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 16:22:09.207812 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 16:22:09.207812 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application self.session.flush() Sep 01 16:22:09.207812 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 16:22:09.207812 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 16:22:09.207812 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 16:22:09.207812 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 16:22:09.207812 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 16:22:09.207812 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application compat.raise_( Sep 01 16:22:09.207812 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 16:22:09.207812 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception Sep 01 16:22:09.207812 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 16:22:09.207812 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 16:22:09.207812 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 16:22:09.207812 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 16:22:09.207812 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 16:22:09.207812 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 16:22:09.208810 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 16:22:09.208810 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 16:22:09.208810 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 01 16:22:09.208810 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 01 16:22:09.208810 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 16:22:09.208810 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 16:22:09.208810 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 16:22:09.208810 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 16:22:09.208810 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 16:22:09.208810 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 16:22:09.208810 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 16:22:09.208810 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 16:22:09.208810 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 16:22:09.208810 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 16:22:09.208810 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 16:22:09.208810 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception Sep 01 16:22:09.209627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 16:22:09.209627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 16:22:09.209627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 16:22:09.209627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 16:22:09.209627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 16:22:09.209627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 16:22:09.209627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 16:22:09.209627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application conn.query(q) Sep 01 16:22:09.209627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 16:22:09.209627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 16:22:09.209627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 16:22:09.209627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application result.read() Sep 01 16:22:09.209627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 16:22:09.209627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 16:22:09.209627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 16:22:09.209627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 16:22:09.211638 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 16:22:09.211638 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 16:22:09.211638 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 16:22:09.211638 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 16:22:09.211638 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 16:22:09.211638 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 16:22:09.211638 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ff7f20fbe3aa498487f3795903840bf5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 16:22:09.211638 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 16:22:09.211638 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:09.211638 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:22:09.211638 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:09.211638 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:09.211638 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:09.211638 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:09.211638 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:09.213867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:09.213867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:09.213867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:09.213867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:09.213867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:09.213867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:09.213867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:09.213867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 01 16:22:09.213867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 01 16:22:09.213867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:09.213867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:09.213867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:22:09.213867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:22:09.213867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:22:09.213867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:22:09.213867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 01 16:22:09.213867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 01 16:22:09.216101 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 01 16:22:09.216101 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 01 16:22:09.216101 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 16:22:09.216101 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 16:22:09.216101 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 16:22:09.216101 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:09.216101 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 492/971] 10.222.0.18 () {60 vars in 1131 bytes} [Thu Sep 1 16:22:08 2022] POST /identity/v3/users => generated 160 bytes in 234 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:22:09.216990 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:09.217100 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ecfe45-a8fa-4886-a901-76860621e5fc None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:09.275900 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 480/972] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:22:08 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:22:09.281243 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:09.282969 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bed76c-c265-481b-b9b8-b2659ee24b06 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:09.327974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ecfe45-a8fa-4886-a901-76860621e5fc None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:09.379736 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c805615-c3b8-46c4-9a63-96ec501a75c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:09.380715 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c805615-c3b8-46c4-9a63-96ec501a75c6 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:09.381214 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c805615-c3b8-46c4-9a63-96ec501a75c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:09.381676 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c805615-c3b8-46c4-9a63-96ec501a75c6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:09.390594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c805615-c3b8-46c4-9a63-96ec501a75c6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:09.392557 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c805615-c3b8-46c4-9a63-96ec501a75c6 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:09.405336 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-3c805615-c3b8-46c4-9a63-96ec501a75c6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:22:09.405336 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:09.405336 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:09.405336 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:09.405336 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:09.405336 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:09.405336 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:09.405336 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:09.405336 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:09.405336 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:09.405336 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:09.405336 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:09.405336 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:22:09.405336 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:22:09.405336 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:22:09.405336 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:22:09.406367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:09.406367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:09.406367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:22:09.406367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:22:09.406367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:22:09.406367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:22:09.406367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:09.406367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:09.406367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:09.406367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:09.406367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:09.406367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:09.406367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:09.406367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:09.406367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:09.406367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:09.406367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:09.406367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:09.407337 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:09.407337 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:09.407337 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:22:09.407337 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:22:09.407337 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:22:09.407337 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:22:09.407337 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:22:09.407337 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:22:09.407337 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:22:09.407337 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:09.410007 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 493/973] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:22:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:22:09.416407 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c805615-c3b8-46c4-9a63-96ec501a75c6 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:09.428890 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bed76c-c265-481b-b9b8-b2659ee24b06 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:09.476184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605fe0d-2cbe-4cf1-aa27-744d38a41737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:09.476826 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605fe0d-2cbe-4cf1-aa27-744d38a41737 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:09.477627 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605fe0d-2cbe-4cf1-aa27-744d38a41737 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:09.477926 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605fe0d-2cbe-4cf1-aa27-744d38a41737 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:09.492458 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4605fe0d-2cbe-4cf1-aa27-744d38a41737 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:09.495041 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4605fe0d-2cbe-4cf1-aa27-744d38a41737 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:09.513299 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-4605fe0d-2cbe-4cf1-aa27-744d38a41737 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:22:09.513299 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:09.513299 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:09.513299 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:09.513299 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:09.513299 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:09.513299 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:09.513299 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:09.513299 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:09.513299 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:09.513299 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:09.513299 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:09.513299 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:22:09.513299 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:22:09.513299 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:22:09.513299 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:22:09.514297 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:09.514297 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:09.514297 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:09.514297 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:09.514297 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:09.514297 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:09.514297 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:09.514297 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:09.514297 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:09.514297 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:09.514297 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:09.514297 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:09.514297 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:09.514297 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:09.514297 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:09.514297 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:09.514297 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:22:09.515181 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:22:09.515181 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:22:09.515181 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:09.517222 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 481/974] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:22:09.522840 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605fe0d-2cbe-4cf1-aa27-744d38a41737 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:09.527600 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:09.535389 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c805615-c3b8-46c4-9a63-96ec501a75c6 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:09.558302 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:09.581887 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2790b4-e546-415e-b6c1-56090595c398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:09.582527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2790b4-e546-415e-b6c1-56090595c398 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:09.582808 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2790b4-e546-415e-b6c1-56090595c398 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:09.583079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2790b4-e546-415e-b6c1-56090595c398 None admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:09.583773 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2790b4-e546-415e-b6c1-56090595c398 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:09.584074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2790b4-e546-415e-b6c1-56090595c398 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:09.585567 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2790b4-e546-415e-b6c1-56090595c398 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:09.639197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 494/975] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Sep 1 16:22:09 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:09.667787 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605fe0d-2cbe-4cf1-aa27-744d38a41737 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:09.706161 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcacda04-4b15-4b00-916f-f5bce13ef267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:09.707197 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacda04-4b15-4b00-916f-f5bce13ef267 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:09.707711 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacda04-4b15-4b00-916f-f5bce13ef267 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:09.708225 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacda04-4b15-4b00-916f-f5bce13ef267 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:09.709128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcacda04-4b15-4b00-916f-f5bce13ef267 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:09.709647 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcacda04-4b15-4b00-916f-f5bce13ef267 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:09.711278 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcacda04-4b15-4b00-916f-f5bce13ef267 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:09.726148 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 482/976] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:22:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:09.735265 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2790b4-e546-415e-b6c1-56090595c398 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:09.795940 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070edd1-49bf-4e85-acda-94a187c74e3d None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:09.796583 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070edd1-49bf-4e85-acda-94a187c74e3d None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:09.797046 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070edd1-49bf-4e85-acda-94a187c74e3d None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:09.799704 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 483/977] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:09.805678 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cddb8e3-d708-492d-88ee-2e25894f9de5 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:09.805967 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cddb8e3-d708-492d-88ee-2e25894f9de5 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:09.806270 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cddb8e3-d708-492d-88ee-2e25894f9de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:09.828359 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-6cddb8e3-d708-492d-88ee-2e25894f9de5 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:09.830278 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6cddb8e3-d708-492d-88ee-2e25894f9de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:09.861911 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2790b4-e546-415e-b6c1-56090595c398 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:09.904664 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110f4aa-4e01-4742-bc03-381ba6cc9b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:09.905225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110f4aa-4e01-4742-bc03-381ba6cc9b15 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:09.905507 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110f4aa-4e01-4742-bc03-381ba6cc9b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:09.905782 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110f4aa-4e01-4742-bc03-381ba6cc9b15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:09.908076 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:09.914422 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a110f4aa-4e01-4742-bc03-381ba6cc9b15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:09.916447 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a110f4aa-4e01-4742-bc03-381ba6cc9b15 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:09.924595 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 484/978] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:22:09.933065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-a110f4aa-4e01-4742-bc03-381ba6cc9b15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:22:09.933065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:09.933065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:09.933065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:09.933065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:09.933065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:09.933065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:09.933065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:09.933065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:09.933065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:09.933065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:09.933065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:09.933065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:22:09.933065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:22:09.933065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:22:09.933968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:22:09.933968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:09.933968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:09.933968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:09.933968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:09.933968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:09.933968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:09.933968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:09.933968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:09.933968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:09.933968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:09.933968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:09.933968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:09.933968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:09.933968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:09.933968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:09.933968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:09.934949 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:22:09.934949 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:22:09.934949 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:22:09.934949 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:22:09.934949 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:22:09.934949 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:22:09.934949 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:22:09.934949 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:09.934949 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 495/979] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:22:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:09.941804 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cddb8e3-d708-492d-88ee-2e25894f9de5 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:09.954036 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0262d37-7f18-4c05-a659-1dc42b47c7bb None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:09.954370 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0262d37-7f18-4c05-a659-1dc42b47c7bb None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:09.954637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0262d37-7f18-4c05-a659-1dc42b47c7bb None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:09.956235 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 496/980] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:09.961045 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e47c60-403c-4ff2-91a0-173142ac1e1d None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:09.961338 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e47c60-403c-4ff2-91a0-173142ac1e1d None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:09.962143 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e47c60-403c-4ff2-91a0-173142ac1e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:09.980819 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-c9e47c60-403c-4ff2-91a0-173142ac1e1d None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:09.982372 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-c9e47c60-403c-4ff2-91a0-173142ac1e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:10.017435 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:10.048056 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cddb8e3-d708-492d-88ee-2e25894f9de5 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:10.062278 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 497/981] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:22:10.081757 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec5f24-b9ee-4f72-9dd4-515fcbd41841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:10.082488 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec5f24-b9ee-4f72-9dd4-515fcbd41841 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:10.082832 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec5f24-b9ee-4f72-9dd4-515fcbd41841 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:10.083169 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec5f24-b9ee-4f72-9dd4-515fcbd41841 None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:10.083920 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec5f24-b9ee-4f72-9dd4-515fcbd41841 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:10.084291 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec5f24-b9ee-4f72-9dd4-515fcbd41841 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:10.085775 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec5f24-b9ee-4f72-9dd4-515fcbd41841 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:10.102735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 485/982] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:22:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:10.104579 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e47c60-403c-4ff2-91a0-173142ac1e1d None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:10.193372 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec5f24-b9ee-4f72-9dd4-515fcbd41841 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:10.205529 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e47c60-403c-4ff2-91a0-173142ac1e1d None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:10.243445 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86ce3b-4d7f-49f4-9e1c-a6d44e21700c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:10.244051 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86ce3b-4d7f-49f4-9e1c-a6d44e21700c None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:10.244599 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86ce3b-4d7f-49f4-9e1c-a6d44e21700c None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:10.244866 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86ce3b-4d7f-49f4-9e1c-a6d44e21700c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:10.245590 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba86ce3b-4d7f-49f4-9e1c-a6d44e21700c None admin] RBAC: Adding query filter params (role.id=a889b69245b44da59d9020a0d39751cb, scope.project.id=36ce80c4e0d64562a9ab3961be76fad9, user.id=a5b3e53fd1314e17ad28c42a44e8eff3) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:10.245896 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba86ce3b-4d7f-49f4-9e1c-a6d44e21700c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:10.247353 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba86ce3b-4d7f-49f4-9e1c-a6d44e21700c None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:10.253187 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 498/983] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:22:10 2022] GET /identity/v3/role_assignments?user.id=a5b3e53fd1314e17ad28c42a44e8eff3&scope.project.id=36ce80c4e0d64562a9ab3961be76fad9&role.id=a889b69245b44da59d9020a0d39751cb => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:10.269309 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:10.323294 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec5f24-b9ee-4f72-9dd4-515fcbd41841 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:10.335859 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86ce3b-4d7f-49f4-9e1c-a6d44e21700c None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:10.361880 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d56301-1d4b-4b35-81a5-b1e57d35a3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:10.362569 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d56301-1d4b-4b35-81a5-b1e57d35a3b8 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:10.365668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d56301-1d4b-4b35-81a5-b1e57d35a3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:10.365668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d56301-1d4b-4b35-81a5-b1e57d35a3b8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:10.365668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d56301-1d4b-4b35-81a5-b1e57d35a3b8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:10.365668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d56301-1d4b-4b35-81a5-b1e57d35a3b8 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:10.375598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:10.376803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-b7d56301-1d4b-4b35-81a5-b1e57d35a3b8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:22:10.376803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:10.376803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:22:10.376803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:22:10.376803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:22:10.376803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:22:10.376803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:22:10.376803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:22:10.376803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:22:10.376803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:10.376803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:22:10.376803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:10.376803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:10.376803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:10.376803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:10.376803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:10.376803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:10.377751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:10.377751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:10.377751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:10.377751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:10.377751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:10.377751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:10.377751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:22:10.377751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:22:10.377751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:22:10.377751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:22:10.377751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:10.377751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:10.377751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:10.377751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:10.377751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:10.377751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:10.378604 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:10.378604 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:10.378604 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:10.378604 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:10.378604 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:10.378604 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:10.378604 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:10.378604 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:10.378604 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:10.378604 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:10.378604 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:22:10.378604 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:22:10.378604 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:22:10.378604 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:10.378604 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 486/984] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Sep 1 16:22:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:10.384092 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d56301-1d4b-4b35-81a5-b1e57d35a3b8 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:10.462389 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86ce3b-4d7f-49f4-9e1c-a6d44e21700c None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:10.490608 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d56301-1d4b-4b35-81a5-b1e57d35a3b8 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:10.499937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa06a3-eb4b-4d87-9b20-0d27911cd698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:10.500561 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa06a3-eb4b-4d87-9b20-0d27911cd698 None admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:10.500845 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa06a3-eb4b-4d87-9b20-0d27911cd698 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:10.501125 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa06a3-eb4b-4d87-9b20-0d27911cd698 None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:10.501789 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa06a3-eb4b-4d87-9b20-0d27911cd698 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:10.503196 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa06a3-eb4b-4d87-9b20-0d27911cd698 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:10.509440 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:10.519756 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 499/985] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Sep 1 16:22:10 2022] POST /identity/v3/services => generated 242 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:22:10.536810 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b7ef7-0f57-4980-b019-53d0de06ba9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:10.537452 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b7ef7-0f57-4980-b019-53d0de06ba9a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:10.537738 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b7ef7-0f57-4980-b019-53d0de06ba9a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:10.538012 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b7ef7-0f57-4980-b019-53d0de06ba9a None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:10.538696 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b7ef7-0f57-4980-b019-53d0de06ba9a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:10.539049 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b7ef7-0f57-4980-b019-53d0de06ba9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:10.541291 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b7ef7-0f57-4980-b019-53d0de06ba9a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:10.548299 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 487/986] 10.222.0.18 () {56 vars in 1122 bytes} [Thu Sep 1 16:22:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:10.553588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa06a3-eb4b-4d87-9b20-0d27911cd698 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:10.656599 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:10.671352 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa06a3-eb4b-4d87-9b20-0d27911cd698 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:10.707511 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23095366-9113-4c89-b6f6-f9926faf46a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:10.708296 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23095366-9113-4c89-b6f6-f9926faf46a8 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:10.708678 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23095366-9113-4c89-b6f6-f9926faf46a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:10.709056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23095366-9113-4c89-b6f6-f9926faf46a8 None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:10.709795 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23095366-9113-4c89-b6f6-f9926faf46a8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:10.710304 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23095366-9113-4c89-b6f6-f9926faf46a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:10.712266 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23095366-9113-4c89-b6f6-f9926faf46a8 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:10.722421 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 500/987] 10.222.0.18 () {56 vars in 1122 bytes} [Thu Sep 1 16:22:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:10.723940 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b7ef7-0f57-4980-b019-53d0de06ba9a None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:10.824874 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b7ef7-0f57-4980-b019-53d0de06ba9a None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:10.859197 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814426e-0847-41bc-8677-1eb7ee3a2b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:10.859827 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814426e-0847-41bc-8677-1eb7ee3a2b05 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:10.860125 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814426e-0847-41bc-8677-1eb7ee3a2b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:10.860370 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814426e-0847-41bc-8677-1eb7ee3a2b05 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:10.861106 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8814426e-0847-41bc-8677-1eb7ee3a2b05 None admin] RBAC: Adding query filter params (interface=admin, service_id=6b871479336649389fd81413c43f513f, region_id=RegionOne) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:10.861395 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8814426e-0847-41bc-8677-1eb7ee3a2b05 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:10.862770 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8814426e-0847-41bc-8677-1eb7ee3a2b05 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:10.870328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 488/988] 10.222.0.18 () {56 vars in 1249 bytes} [Thu Sep 1 16:22:10 2022] GET /identity/v3/endpoints?service_id=6b871479336649389fd81413c43f513f&interface=admin®ion_id=RegionOne => generated 193 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:10.873907 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23095366-9113-4c89-b6f6-f9926faf46a8 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:10.974760 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23095366-9113-4c89-b6f6-f9926faf46a8 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:11.007328 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef00714-b2f5-4bae-b5f7-1e91252bacdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:11.007937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef00714-b2f5-4bae-b5f7-1e91252bacdc None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:11.008216 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef00714-b2f5-4bae-b5f7-1e91252bacdc None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:11.008996 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef00714-b2f5-4bae-b5f7-1e91252bacdc None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:11.009627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef00714-b2f5-4bae-b5f7-1e91252bacdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:11.011011 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef00714-b2f5-4bae-b5f7-1e91252bacdc None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:11.022465 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 501/989] 10.222.0.18 () {56 vars in 1087 bytes} [Thu Sep 1 16:22:10 2022] GET /identity/v3/services => generated 1727 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:22:12.180795 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3427c-5e5a-4d0f-bc7d-8058d4663a87 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:12.181314 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3427c-5e5a-4d0f-bc7d-8058d4663a87 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:12.181423 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3427c-5e5a-4d0f-bc7d-8058d4663a87 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:12.182897 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 489/990] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:12.187786 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59913204-9f80-4cbb-83ce-d9e1a343e711 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:12.188160 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59913204-9f80-4cbb-83ce-d9e1a343e711 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:12.188355 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59913204-9f80-4cbb-83ce-d9e1a343e711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:12.206490 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-59913204-9f80-4cbb-83ce-d9e1a343e711 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:12.207679 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-59913204-9f80-4cbb-83ce-d9e1a343e711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:12.221338 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:12.276892 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 502/991] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:12 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:22:12.436998 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5053a-55a2-4c62-b709-20f229300b1e None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:12.437189 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5053a-55a2-4c62-b709-20f229300b1e None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:12.437472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5053a-55a2-4c62-b709-20f229300b1e None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:12.439002 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 490/992] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:12.443972 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0e1a4-aae2-4bf7-806d-2ad4eb37578e None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:12.444269 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0e1a4-aae2-4bf7-806d-2ad4eb37578e None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:12.444549 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0e1a4-aae2-4bf7-806d-2ad4eb37578e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:12.463883 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-b8d0e1a4-aae2-4bf7-806d-2ad4eb37578e None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:12.465068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-b8d0e1a4-aae2-4bf7-806d-2ad4eb37578e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:12.533174 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 503/993] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:12 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:22:12.542575 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5053a-55a2-4c62-b709-20f229300b1e None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:12.585945 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:12.647544 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5053a-55a2-4c62-b709-20f229300b1e None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:12.681551 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5dcd9e-a7dd-4b1f-bfb3-ae16e684ac86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:12.682183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5dcd9e-a7dd-4b1f-bfb3-ae16e684ac86 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:12.682490 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5dcd9e-a7dd-4b1f-bfb3-ae16e684ac86 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:12.682767 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5dcd9e-a7dd-4b1f-bfb3-ae16e684ac86 None admin] PATH_INFO: `/v3/services/image` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:12.683349 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5dcd9e-a7dd-4b1f-bfb3-ae16e684ac86 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:12.684709 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5dcd9e-a7dd-4b1f-bfb3-ae16e684ac86 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:12.693626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-6a5dcd9e-a7dd-4b1f-bfb3-ae16e684ac86 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:22:12.693626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:12.693626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:22:12.693626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:22:12.693626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:22:12.693626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:22:12.693626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:22:12.693626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:22:12.693626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:22:12.693626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:12.693626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:22:12.693626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:12.693626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:12.693626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:12.693626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:12.693626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:12.693626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:12.694806 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:12.694806 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:12.694806 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:12.694806 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:12.694806 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:12.694806 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:12.694806 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:22:12.694806 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:22:12.694806 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:22:12.694806 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:22:12.694806 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:12.694806 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:12.694806 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:12.694806 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:12.694806 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:12.694806 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:12.695603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:12.695603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:12.695603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:12.695603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:12.695603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:12.695603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:12.695603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:12.695603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:12.695603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:12.695603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:12.695603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:22:12.695603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:22:12.695603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:22:12.695603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:12.695603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 491/994] 10.222.0.18 () {56 vars in 1105 bytes} [Thu Sep 1 16:22:12 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:12.702435 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d0e1a4-aae2-4bf7-806d-2ad4eb37578e None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:12.780385 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:12.803004 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5dcd9e-a7dd-4b1f-bfb3-ae16e684ac86 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:12.806353 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d0e1a4-aae2-4bf7-806d-2ad4eb37578e None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:12.845094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f088aaa-7c22-49af-86bd-21aa1c19356c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:12.845702 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f088aaa-7c22-49af-86bd-21aa1c19356c None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:12.845981 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f088aaa-7c22-49af-86bd-21aa1c19356c None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:12.846275 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f088aaa-7c22-49af-86bd-21aa1c19356c None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:12.847265 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f088aaa-7c22-49af-86bd-21aa1c19356c None admin] RBAC: Adding query filter params (name=image) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:12.847572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f088aaa-7c22-49af-86bd-21aa1c19356c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:12.849258 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f088aaa-7c22-49af-86bd-21aa1c19356c None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:12.855362 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 504/995] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:12.861979 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f088aaa-7c22-49af-86bd-21aa1c19356c None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:12.913376 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5dcd9e-a7dd-4b1f-bfb3-ae16e684ac86 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:12.917193 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:12.951041 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1cf3d9-645b-4210-8c97-4e4c30de548c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:12.951602 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1cf3d9-645b-4210-8c97-4e4c30de548c None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:12.951885 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1cf3d9-645b-4210-8c97-4e4c30de548c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:12.952160 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1cf3d9-645b-4210-8c97-4e4c30de548c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:12.952746 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1cf3d9-645b-4210-8c97-4e4c30de548c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:12.954108 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1cf3d9-645b-4210-8c97-4e4c30de548c None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:12.963228 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-5e1cf3d9-645b-4210-8c97-4e4c30de548c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:22:12.963228 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:12.963228 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:22:12.963228 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:22:12.963228 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:22:12.963228 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:22:12.963228 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:22:12.963228 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:22:12.963228 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:22:12.963228 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:12.963228 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:22:12.963228 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:12.963228 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:12.963228 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:12.963228 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:12.963228 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:12.963228 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:12.964385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:12.964385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:12.964385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:12.964385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:12.964385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:12.964385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:12.964385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:22:12.964385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:22:12.964385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:22:12.964385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:22:12.964385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:12.964385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:12.964385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:12.964385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:12.964385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:12.964385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:12.965211 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:12.965211 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:12.965211 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:12.965211 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:12.965211 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:12.965211 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:12.965211 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:12.965211 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:12.965211 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:12.965211 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:12.965211 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:22:12.965211 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:22:12.965211 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:22:12.965211 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:12.965211 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 492/996] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:22:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:22:12.966016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f088aaa-7c22-49af-86bd-21aa1c19356c None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:12.966091 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096d458-0ac6-4e3c-a281-0e032bf23231 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:12.966091 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096d458-0ac6-4e3c-a281-0e032bf23231 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:12.966091 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096d458-0ac6-4e3c-a281-0e032bf23231 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:12.971202 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 493/997] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:12.971202 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b096d458-0ac6-4e3c-a281-0e032bf23231 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:12.998896 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7d284-4dad-428b-88db-89776d55909b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:12.999500 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7d284-4dad-428b-88db-89776d55909b None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:12.999786 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7d284-4dad-428b-88db-89776d55909b None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:13.000061 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7d284-4dad-428b-88db-89776d55909b None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:13.000686 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf7d284-4dad-428b-88db-89776d55909b None admin] RBAC: Adding query filter params (type=image) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:13.000994 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf7d284-4dad-428b-88db-89776d55909b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:13.002382 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf7d284-4dad-428b-88db-89776d55909b None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:13.006663 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 505/998] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:12 2022] GET /identity/v3/services?type=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:13.007789 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c718baf-49ba-4135-8c9b-74530c8bb27d None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:13.008076 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c718baf-49ba-4135-8c9b-74530c8bb27d None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:13.008381 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c718baf-49ba-4135-8c9b-74530c8bb27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:13.023280 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:13.030469 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-9c718baf-49ba-4135-8c9b-74530c8bb27d None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:13.032238 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-9c718baf-49ba-4135-8c9b-74530c8bb27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:13.092612 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b096d458-0ac6-4e3c-a281-0e032bf23231 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:13.112732 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 506/999] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:13 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:22:13.136340 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c386b3c-1f93-493d-9747-7c3df1072b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:13.137060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c386b3c-1f93-493d-9747-7c3df1072b89 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:13.137454 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c386b3c-1f93-493d-9747-7c3df1072b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:13.137793 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c386b3c-1f93-493d-9747-7c3df1072b89 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:13.138591 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c386b3c-1f93-493d-9747-7c3df1072b89 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:13.138999 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c386b3c-1f93-493d-9747-7c3df1072b89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:13.140492 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c386b3c-1f93-493d-9747-7c3df1072b89 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:13.146276 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 494/1000] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Sep 1 16:22:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:13.153066 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c718baf-49ba-4135-8c9b-74530c8bb27d None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:13.258094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c718baf-49ba-4135-8c9b-74530c8bb27d None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:13.292358 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d72f5bc-5d47-4fbb-8f86-9d125bf46ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:13.292967 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72f5bc-5d47-4fbb-8f86-9d125bf46ece None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:13.293252 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72f5bc-5d47-4fbb-8f86-9d125bf46ece None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:13.293529 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72f5bc-5d47-4fbb-8f86-9d125bf46ece None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:13.294163 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d72f5bc-5d47-4fbb-8f86-9d125bf46ece None admin] RBAC: Adding query filter params (type=compute) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:13.294495 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d72f5bc-5d47-4fbb-8f86-9d125bf46ece None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:13.295873 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d72f5bc-5d47-4fbb-8f86-9d125bf46ece None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:13.297933 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:13.306316 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 507/1001] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Sep 1 16:22:13 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:13.307550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c386b3c-1f93-493d-9747-7c3df1072b89 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:13.340785 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:13.381674 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d72f5bc-5d47-4fbb-8f86-9d125bf46ece None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:13.422637 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c386b3c-1f93-493d-9747-7c3df1072b89 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:13.465827 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b35db80-cd61-453c-9f7a-fc0709084f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:13.466472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b35db80-cd61-453c-9f7a-fc0709084f91 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:13.466757 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b35db80-cd61-453c-9f7a-fc0709084f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:13.467069 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b35db80-cd61-453c-9f7a-fc0709084f91 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:13.467730 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b35db80-cd61-453c-9f7a-fc0709084f91 None admin] RBAC: Adding query filter params (interface=public, service_id=9b8af7c6bc6c4e149e1dea9ec7553629, region_id=RegionOne) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:13.468214 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b35db80-cd61-453c-9f7a-fc0709084f91 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:13.469384 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b35db80-cd61-453c-9f7a-fc0709084f91 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:13.474614 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 495/1002] 10.222.0.18 () {56 vars in 1251 bytes} [Thu Sep 1 16:22:13 2022] GET /identity/v3/endpoints?service_id=9b8af7c6bc6c4e149e1dea9ec7553629&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:13.480016 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b35db80-cd61-453c-9f7a-fc0709084f91 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:13.490349 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d72f5bc-5d47-4fbb-8f86-9d125bf46ece None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:13.525219 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d865791-0273-444f-bce5-d4643536920c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:13.525779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d865791-0273-444f-bce5-d4643536920c None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:13.526061 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d865791-0273-444f-bce5-d4643536920c None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:13.526365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d865791-0273-444f-bce5-d4643536920c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:13.526953 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d865791-0273-444f-bce5-d4643536920c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:13.528317 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d865791-0273-444f-bce5-d4643536920c None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:13.537190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-6d865791-0273-444f-bce5-d4643536920c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:22:13.537190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:13.537190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:22:13.537190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:22:13.537190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:22:13.537190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:22:13.537190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:22:13.537190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:22:13.537190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:22:13.537190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:13.537190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:22:13.537190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:13.537190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:13.537190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:13.537190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:13.537190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:13.537190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:13.538167 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:13.538167 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:13.538167 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:13.538167 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:13.538167 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:13.538167 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:13.538167 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:22:13.538167 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:22:13.538167 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:22:13.538167 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:22:13.538167 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:13.538167 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:13.538167 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:13.538167 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:13.538167 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:13.538167 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:13.539065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:13.539065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:13.539065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:13.539065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:13.539065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:13.539065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:13.539065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:13.539065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:13.539065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:13.539065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:13.539065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:22:13.539065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:22:13.539065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:22:13.539065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:13.539065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 508/1003] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Sep 1 16:22:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:13.544050 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d865791-0273-444f-bce5-d4643536920c None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:13.587776 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:13.594288 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b35db80-cd61-453c-9f7a-fc0709084f91 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:13.611930 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:13.630094 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5671a97-ff35-4f22-b09f-260b2cf712b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:13.630817 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5671a97-ff35-4f22-b09f-260b2cf712b2 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:13.631101 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5671a97-ff35-4f22-b09f-260b2cf712b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:13.631372 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5671a97-ff35-4f22-b09f-260b2cf712b2 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:13.632349 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5671a97-ff35-4f22-b09f-260b2cf712b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:13.634670 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5671a97-ff35-4f22-b09f-260b2cf712b2 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:13.646272 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 496/1004] 10.222.0.18 () {56 vars in 1087 bytes} [Thu Sep 1 16:22:13 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:22:13.660916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d865791-0273-444f-bce5-d4643536920c None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:13.700150 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99acfcdd-4da8-400c-9f12-d6580d157afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:13.700780 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99acfcdd-4da8-400c-9f12-d6580d157afe None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:13.701067 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99acfcdd-4da8-400c-9f12-d6580d157afe None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:13.701344 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99acfcdd-4da8-400c-9f12-d6580d157afe None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:13.701993 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99acfcdd-4da8-400c-9f12-d6580d157afe None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:13.702335 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99acfcdd-4da8-400c-9f12-d6580d157afe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:13.703742 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99acfcdd-4da8-400c-9f12-d6580d157afe None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:13.709204 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 509/1005] 10.222.0.18 () {56 vars in 1122 bytes} [Thu Sep 1 16:22:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:13.715204 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5671a97-ff35-4f22-b09f-260b2cf712b2 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:13.819787 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5671a97-ff35-4f22-b09f-260b2cf712b2 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:13.854737 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b1d86-3c26-44b6-8a82-a7f7a389ae05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:13.855354 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b1d86-3c26-44b6-8a82-a7f7a389ae05 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:13.855637 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b1d86-3c26-44b6-8a82-a7f7a389ae05 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:13.855911 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b1d86-3c26-44b6-8a82-a7f7a389ae05 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:13.856538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620b1d86-3c26-44b6-8a82-a7f7a389ae05 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:13.856843 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620b1d86-3c26-44b6-8a82-a7f7a389ae05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:13.858200 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620b1d86-3c26-44b6-8a82-a7f7a389ae05 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:13.863687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 497/1006] 10.222.0.18 () {56 vars in 1122 bytes} [Thu Sep 1 16:22:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:13.870639 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99acfcdd-4da8-400c-9f12-d6580d157afe None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:13.972015 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99acfcdd-4da8-400c-9f12-d6580d157afe None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:13.998998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:14.008159 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e6f2b-1476-445d-b3cf-a7b5938ebe50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:14.008777 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e6f2b-1476-445d-b3cf-a7b5938ebe50 None admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:14.009061 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e6f2b-1476-445d-b3cf-a7b5938ebe50 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:14.009338 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e6f2b-1476-445d-b3cf-a7b5938ebe50 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:14.010020 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643e6f2b-1476-445d-b3cf-a7b5938ebe50 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:14.011426 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643e6f2b-1476-445d-b3cf-a7b5938ebe50 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:14.030481 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 510/1007] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:22:13 2022] POST /identity/v3/endpoints => generated 325 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:22:14.912855 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1b948-5edc-452f-b5ee-a863b1abb430 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:14.913404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1b948-5edc-452f-b5ee-a863b1abb430 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:14.913404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1b948-5edc-452f-b5ee-a863b1abb430 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:14.914926 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 498/1008] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:14.919861 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fecba6e-194b-4491-9745-9a9e7d008317 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:14.920237 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fecba6e-194b-4491-9745-9a9e7d008317 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:14.920407 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fecba6e-194b-4491-9745-9a9e7d008317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:14.938684 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-0fecba6e-194b-4491-9745-9a9e7d008317 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:14.939858 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-0fecba6e-194b-4491-9745-9a9e7d008317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:15.015104 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 511/1009] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:14 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:22:15.284048 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1b948-5edc-452f-b5ee-a863b1abb430 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:15.312609 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:15.402939 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1b948-5edc-452f-b5ee-a863b1abb430 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:15.437616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b76e5c-4151-4964-93e5-b4ff7f831303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:15.438227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b76e5c-4151-4964-93e5-b4ff7f831303 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:15.438533 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b76e5c-4151-4964-93e5-b4ff7f831303 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:15.438805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b76e5c-4151-4964-93e5-b4ff7f831303 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:15.439461 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b76e5c-4151-4964-93e5-b4ff7f831303 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:15.440829 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b76e5c-4151-4964-93e5-b4ff7f831303 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:15.456985 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 499/1010] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Sep 1 16:22:15 2022] POST /identity/v3/services => generated 242 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:22:15.582786 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc4598-cc0d-42de-a8cd-e668060ef42d None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:15.583034 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc4598-cc0d-42de-a8cd-e668060ef42d None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:15.583474 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc4598-cc0d-42de-a8cd-e668060ef42d None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:15.585220 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 512/1011] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:15.590560 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca2634-353d-4f55-b41f-c296bb7f4d1f None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:15.590858 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca2634-353d-4f55-b41f-c296bb7f4d1f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:15.591155 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca2634-353d-4f55-b41f-c296bb7f4d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:15.610412 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-2aca2634-353d-4f55-b41f-c296bb7f4d1f None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:15.611669 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2aca2634-353d-4f55-b41f-c296bb7f4d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:15.705005 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 500/1012] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:15 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:22:15.926596 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9814a-8e83-4374-8b5c-f9a3d0325130 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:15.927087 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9814a-8e83-4374-8b5c-f9a3d0325130 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:15.927087 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9814a-8e83-4374-8b5c-f9a3d0325130 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:15.928370 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 513/1013] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:15.934362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6461a1d-d1ef-43b6-bb87-f5214079d429 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:15.934755 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6461a1d-d1ef-43b6-bb87-f5214079d429 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:15.935041 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6461a1d-d1ef-43b6-bb87-f5214079d429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:15.953735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-f6461a1d-d1ef-43b6-bb87-f5214079d429 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:15.954937 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f6461a1d-d1ef-43b6-bb87-f5214079d429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:15.957779 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:15.977067 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9814a-8e83-4374-8b5c-f9a3d0325130 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:16.025616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 501/1014] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:15 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:22:16.091717 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9814a-8e83-4374-8b5c-f9a3d0325130 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:16.105837 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:16.128604 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0459fc-f491-4f8c-ae99-46087d46cb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:16.129230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0459fc-f491-4f8c-ae99-46087d46cb2c None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:16.129510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0459fc-f491-4f8c-ae99-46087d46cb2c None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:16.129781 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0459fc-f491-4f8c-ae99-46087d46cb2c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:16.130384 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0459fc-f491-4f8c-ae99-46087d46cb2c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:16.131754 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0459fc-f491-4f8c-ae99-46087d46cb2c None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:16.141122 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-9a0459fc-f491-4f8c-ae99-46087d46cb2c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:22:16.141122 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:16.141122 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:22:16.141122 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:22:16.141122 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:22:16.141122 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:22:16.141122 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:22:16.141122 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:22:16.141122 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:22:16.141122 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:16.141122 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:22:16.141122 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:16.141122 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:16.141122 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:16.141122 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:16.141122 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:16.141122 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:16.142135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:16.142135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:16.142135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:16.142135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:16.142135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:16.142135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:16.142135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:22:16.142135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:22:16.142135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:22:16.142135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:22:16.142135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:16.142135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:16.142135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:16.142135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:16.142135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:16.142135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:16.143676 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:16.143676 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:16.143676 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:16.143676 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:16.143676 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:16.143676 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:16.143676 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:16.143676 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:16.143676 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:16.143676 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:16.143676 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:22:16.143676 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:22:16.143676 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:22:16.143676 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:16.143676 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 514/1015] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:22:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:22:16.151529 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6461a1d-d1ef-43b6-bb87-f5214079d429 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:16.251467 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6461a1d-d1ef-43b6-bb87-f5214079d429 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:16.286761 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac36f0-c4c5-4844-b590-0477dde5d88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:16.287360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac36f0-c4c5-4844-b590-0477dde5d88e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:16.287640 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac36f0-c4c5-4844-b590-0477dde5d88e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:16.287921 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac36f0-c4c5-4844-b590-0477dde5d88e None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:16.288597 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac36f0-c4c5-4844-b590-0477dde5d88e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:16.288904 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac36f0-c4c5-4844-b590-0477dde5d88e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:16.290799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac36f0-c4c5-4844-b590-0477dde5d88e None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:16.297695 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0459fc-f491-4f8c-ae99-46087d46cb2c None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:16.298607 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 502/1016] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Sep 1 16:22:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:16.306035 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac36f0-c4c5-4844-b590-0477dde5d88e None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:16.310886 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:16.416344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac36f0-c4c5-4844-b590-0477dde5d88e None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:16.419323 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0459fc-f491-4f8c-ae99-46087d46cb2c None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:16.453976 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b454155-a769-4e56-88ee-54446cefe29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:16.454941 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b454155-a769-4e56-88ee-54446cefe29b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:16.455400 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b454155-a769-4e56-88ee-54446cefe29b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:16.455863 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b454155-a769-4e56-88ee-54446cefe29b None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:16.456674 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b454155-a769-4e56-88ee-54446cefe29b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:16.457163 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b454155-a769-4e56-88ee-54446cefe29b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:16.457645 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a36f8-a7ef-4cb1-9c08-7d417dcd8888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:16.458312 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a36f8-a7ef-4cb1-9c08-7d417dcd8888 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:16.458602 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a36f8-a7ef-4cb1-9c08-7d417dcd8888 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:16.458716 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b454155-a769-4e56-88ee-54446cefe29b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:16.458885 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a36f8-a7ef-4cb1-9c08-7d417dcd8888 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:16.459998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2a36f8-a7ef-4cb1-9c08-7d417dcd8888 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:16.461463 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2a36f8-a7ef-4cb1-9c08-7d417dcd8888 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:16.468221 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 515/1017] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:16.468389 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 503/1018] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Sep 1 16:22:16 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:16.469052 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b454155-a769-4e56-88ee-54446cefe29b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:16.469163 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a36f8-a7ef-4cb1-9c08-7d417dcd8888 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:16.517784 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:16.541378 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:16.579258 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a36f8-a7ef-4cb1-9c08-7d417dcd8888 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:16.579622 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b454155-a769-4e56-88ee-54446cefe29b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:16.614718 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf68d60c-2c3a-43a4-805e-583c82474ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:16.615329 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68d60c-2c3a-43a4-805e-583c82474ac0 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:16.615608 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68d60c-2c3a-43a4-805e-583c82474ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:16.615880 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68d60c-2c3a-43a4-805e-583c82474ac0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:16.616539 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf68d60c-2c3a-43a4-805e-583c82474ac0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:16.617619 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64f6eb-7d56-465a-b878-285ecda0cd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:16.617927 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf68d60c-2c3a-43a4-805e-583c82474ac0 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:16.618228 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64f6eb-7d56-465a-b878-285ecda0cd86 None admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:16.618534 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64f6eb-7d56-465a-b878-285ecda0cd86 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:16.618805 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64f6eb-7d56-465a-b878-285ecda0cd86 None admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:16.619719 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64f6eb-7d56-465a-b878-285ecda0cd86 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:16.621296 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64f6eb-7d56-465a-b878-285ecda0cd86 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:16.638526 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 504/1019] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:22:16 2022] POST /identity/v3/endpoints => generated 334 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:22:16.654624 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 516/1020] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Sep 1 16:22:16 2022] POST /identity/v3/projects => generated 311 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:22:17.386897 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b2651-8a86-4c66-9f30-87bd85c1ca80 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:17.387336 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b2651-8a86-4c66-9f30-87bd85c1ca80 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:17.387530 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b2651-8a86-4c66-9f30-87bd85c1ca80 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:17.388943 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 505/1021] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:17.393818 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac3dea-b977-4df4-9713-67decb3ace60 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:17.394222 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac3dea-b977-4df4-9713-67decb3ace60 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:17.394439 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac3dea-b977-4df4-9713-67decb3ace60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:17.418612 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-42ac3dea-b977-4df4-9713-67decb3ace60 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:17.419812 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-42ac3dea-b977-4df4-9713-67decb3ace60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:17.499287 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 517/1022] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:22:17.766595 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393b2651-8a86-4c66-9f30-87bd85c1ca80 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:17.873683 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393b2651-8a86-4c66-9f30-87bd85c1ca80 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:17.896610 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:17.908754 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484d6b1-52ec-4dc6-9cbb-8b4e51c8cefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:17.909386 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484d6b1-52ec-4dc6-9cbb-8b4e51c8cefc None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:17.909667 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484d6b1-52ec-4dc6-9cbb-8b4e51c8cefc None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:17.909938 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484d6b1-52ec-4dc6-9cbb-8b4e51c8cefc None admin] PATH_INFO: `/v3/services/image` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:17.910543 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0484d6b1-52ec-4dc6-9cbb-8b4e51c8cefc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:17.911900 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0484d6b1-52ec-4dc6-9cbb-8b4e51c8cefc None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:17.920605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-0484d6b1-52ec-4dc6-9cbb-8b4e51c8cefc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:22:17.920605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:17.920605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:22:17.920605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:22:17.920605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:22:17.920605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:22:17.920605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:22:17.920605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:22:17.920605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:22:17.920605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:17.920605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:22:17.920605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:17.920605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:17.920605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:17.920605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:17.920605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:17.920605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:17.921568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:17.921568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:17.921568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:17.921568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:17.921568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:17.921568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:17.921568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:22:17.921568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:22:17.921568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:22:17.921568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:22:17.921568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:17.921568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:17.921568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:17.921568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:17.921568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:17.921568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:17.922411 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:17.922411 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:17.922411 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:17.922411 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:17.922411 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:17.922411 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:17.922411 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:17.922411 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:17.922411 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:17.922411 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:17.922411 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:22:17.922411 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:22:17.922411 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:22:17.922411 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:17.923885 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 506/1023] 10.222.0.18 () {56 vars in 1105 bytes} [Thu Sep 1 16:22:17 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:17.927565 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac3dea-b977-4df4-9713-67decb3ace60 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:17.952262 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:18.027790 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac3dea-b977-4df4-9713-67decb3ace60 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:18.061252 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b573b34-42bb-4e16-a922-f734d76cc14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:18.061870 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b573b34-42bb-4e16-a922-f734d76cc14f None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:18.062153 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b573b34-42bb-4e16-a922-f734d76cc14f None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:18.062454 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b573b34-42bb-4e16-a922-f734d76cc14f None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:18.063085 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b573b34-42bb-4e16-a922-f734d76cc14f None admin] RBAC: Adding query filter params (name=image) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:18.063389 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b573b34-42bb-4e16-a922-f734d76cc14f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:18.064758 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b573b34-42bb-4e16-a922-f734d76cc14f None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:18.069827 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 518/1024] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:18.075841 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484d6b1-52ec-4dc6-9cbb-8b4e51c8cefc None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:18.174582 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484d6b1-52ec-4dc6-9cbb-8b4e51c8cefc None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:18.208895 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010dcc8-feb1-49a5-b84f-b0c68a376939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:18.209511 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010dcc8-feb1-49a5-b84f-b0c68a376939 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:18.209793 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010dcc8-feb1-49a5-b84f-b0c68a376939 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:18.210066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010dcc8-feb1-49a5-b84f-b0c68a376939 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:18.210736 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5010dcc8-feb1-49a5-b84f-b0c68a376939 None admin] RBAC: Adding query filter params (type=image) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:18.211079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5010dcc8-feb1-49a5-b84f-b0c68a376939 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:18.212436 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5010dcc8-feb1-49a5-b84f-b0c68a376939 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:18.217480 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 507/1025] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:18 2022] GET /identity/v3/services?type=image => generated 350 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:18.223205 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b573b34-42bb-4e16-a922-f734d76cc14f None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:18.311509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:18.324888 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b573b34-42bb-4e16-a922-f734d76cc14f None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:18.365448 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e717f-c3ee-4d89-a78f-01779d89abe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:18.365448 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e717f-c3ee-4d89-a78f-01779d89abe0 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:18.365448 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e717f-c3ee-4d89-a78f-01779d89abe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:18.365448 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e717f-c3ee-4d89-a78f-01779d89abe0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:18.365864 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9e717f-c3ee-4d89-a78f-01779d89abe0 None admin] RBAC: Adding query filter params (interface=public, service_id=9987f139bc504d419a5ce06f1b8463a1, region_id=RegionOne) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:18.365864 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9e717f-c3ee-4d89-a78f-01779d89abe0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:18.368532 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9e717f-c3ee-4d89-a78f-01779d89abe0 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:18.373703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 519/1026] 10.222.0.18 () {56 vars in 1251 bytes} [Thu Sep 1 16:22:18 2022] GET /identity/v3/endpoints?service_id=9987f139bc504d419a5ce06f1b8463a1&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:18.379059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010dcc8-feb1-49a5-b84f-b0c68a376939 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:18.432307 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:18.480473 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010dcc8-feb1-49a5-b84f-b0c68a376939 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:18.514648 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e1d615-9fa6-4b14-826c-441e4c82e630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:18.515263 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1d615-9fa6-4b14-826c-441e4c82e630 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:18.515546 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1d615-9fa6-4b14-826c-441e4c82e630 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:18.515821 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1d615-9fa6-4b14-826c-441e4c82e630 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:18.516430 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e1d615-9fa6-4b14-826c-441e4c82e630 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:18.517793 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e1d615-9fa6-4b14-826c-441e4c82e630 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:18.530312 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 508/1027] 10.222.0.18 () {56 vars in 1087 bytes} [Thu Sep 1 16:22:18 2022] GET /identity/v3/services => generated 1957 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:22:18.618315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40d927-0924-4e1f-9cd5-ea793ff436d7 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:18.619669 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40d927-0924-4e1f-9cd5-ea793ff436d7 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:18.619669 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40d927-0924-4e1f-9cd5-ea793ff436d7 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:18.621110 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 520/1028] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:18.626588 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b258a-fc0d-46e3-bca5-cdce88071178 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:18.626880 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b258a-fc0d-46e3-bca5-cdce88071178 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:18.627159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b258a-fc0d-46e3-bca5-cdce88071178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:18.645974 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-bd3b258a-fc0d-46e3-bca5-cdce88071178 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:18.647212 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-bd3b258a-fc0d-46e3-bca5-cdce88071178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:18.653215 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0c8b5-d9cb-4660-9094-bb46a815a0c1 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:18.653503 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0c8b5-d9cb-4660-9094-bb46a815a0c1 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:18.653788 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0c8b5-d9cb-4660-9094-bb46a815a0c1 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:18.655307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 521/1029] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:18.660229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf9e8f-e0af-4c69-9c5a-4efd6321e699 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:18.660571 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf9e8f-e0af-4c69-9c5a-4efd6321e699 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:18.660829 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf9e8f-e0af-4c69-9c5a-4efd6321e699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:18.681420 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-42cf9e8f-e0af-4c69-9c5a-4efd6321e699 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:18.682659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-42cf9e8f-e0af-4c69-9c5a-4efd6321e699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:18.720721 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 509/1030] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:22:18.755664 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 522/1031] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:22:18.991253 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b258a-fc0d-46e3-bca5-cdce88071178 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:19.022704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf9e8f-e0af-4c69-9c5a-4efd6321e699 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:19.040965 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:19.098972 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b258a-fc0d-46e3-bca5-cdce88071178 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:19.107515 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:19.126533 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf9e8f-e0af-4c69-9c5a-4efd6321e699 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:19.138031 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd2531-2326-4c00-9e60-74b562599a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:19.138730 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd2531-2326-4c00-9e60-74b562599a06 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:19.139023 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd2531-2326-4c00-9e60-74b562599a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:19.139273 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd2531-2326-4c00-9e60-74b562599a06 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:19.140384 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd2531-2326-4c00-9e60-74b562599a06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:19.141816 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd2531-2326-4c00-9e60-74b562599a06 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:19.143773 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 510/1032] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:19.149558 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd2531-2326-4c00-9e60-74b562599a06 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:19.167260 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b02dfd-7293-4dcf-a7fd-ef3ba1230ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:19.167892 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b02dfd-7293-4dcf-a7fd-ef3ba1230ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:19.168172 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b02dfd-7293-4dcf-a7fd-ef3ba1230ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:19.168507 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b02dfd-7293-4dcf-a7fd-ef3ba1230ae5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:19.179826 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b02dfd-7293-4dcf-a7fd-ef3ba1230ae5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:19.184594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-54b02dfd-7293-4dcf-a7fd-ef3ba1230ae5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:22:19.184594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:19.184594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:19.184594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:19.184594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:19.184594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:19.184594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:19.184594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:19.184594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:19.184594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:19.184594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:19.184594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:19.184594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:19.184594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:19.184594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:19.184594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:19.185513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:19.185513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:19.185513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:19.185513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:19.185513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:19.185513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:19.185513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:19.185513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:19.185513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:19.185513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:19.185513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:19.185513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:19.185513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:19.185513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:19.185513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:19.185513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:19.186384 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:19.186384 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:19.186384 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:19.186384 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:19.186384 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:19.186384 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:19.186384 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:19.186384 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:19.186384 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:19.186384 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:19.186384 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:22:19.186384 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:19.186384 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 523/1033] 10.222.0.18 () {56 vars in 1120 bytes} [Thu Sep 1 16:22:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:22:19.191364 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b02dfd-7293-4dcf-a7fd-ef3ba1230ae5 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:19.254732 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd2531-2326-4c00-9e60-74b562599a06 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:19.270867 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:19.296367 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98b8b6-e974-4fda-b7d3-a507263b8a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:19.296717 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98b8b6-e974-4fda-b7d3-a507263b8a97 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:19.296994 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98b8b6-e974-4fda-b7d3-a507263b8a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:19.297267 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98b8b6-e974-4fda-b7d3-a507263b8a97 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:19.297906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f98b8b6-e974-4fda-b7d3-a507263b8a97 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:19.299348 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f98b8b6-e974-4fda-b7d3-a507263b8a97 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:19.299450 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b02dfd-7293-4dcf-a7fd-ef3ba1230ae5 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:19.324232 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 511/1034] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 1 16:22:19 2022] POST /identity/v3/users => generated 285 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:22:19.335729 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9063803-1a3a-4691-af38-88c18f1b0a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:19.337025 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9063803-1a3a-4691-af38-88c18f1b0a13 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:19.337308 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9063803-1a3a-4691-af38-88c18f1b0a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:19.337580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9063803-1a3a-4691-af38-88c18f1b0a13 None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:19.338297 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9063803-1a3a-4691-af38-88c18f1b0a13 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:19.338607 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9063803-1a3a-4691-af38-88c18f1b0a13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:19.340029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9063803-1a3a-4691-af38-88c18f1b0a13 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:19.354440 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 524/1035] 10.222.0.18 () {56 vars in 1115 bytes} [Thu Sep 1 16:22:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:19.356346 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98b8b6-e974-4fda-b7d3-a507263b8a97 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:19.467650 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98b8b6-e974-4fda-b7d3-a507263b8a97 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:19.503487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f12e83-972f-4d62-b097-50e8002dc7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:19.504133 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f12e83-972f-4d62-b097-50e8002dc7b2 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:19.504411 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f12e83-972f-4d62-b097-50e8002dc7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:19.504683 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f12e83-972f-4d62-b097-50e8002dc7b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:19.515961 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f12e83-972f-4d62-b097-50e8002dc7b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:19.517325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f12e83-972f-4d62-b097-50e8002dc7b2 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:19.532326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-03f12e83-972f-4d62-b097-50e8002dc7b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:22:19.532326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:19.532326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:19.532326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:19.532326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:19.532326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:19.532326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:19.532326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:19.532326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:19.532326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:19.532326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:19.532326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:19.532326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:22:19.532326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:22:19.532326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:22:19.532326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:22:19.533283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:19.533283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:19.533283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:19.533283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:19.533283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:19.533283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:19.533283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:19.533283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:19.533283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:19.533283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:19.533283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:19.533283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:19.533283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:19.533283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:19.533283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:19.533283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:19.533283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:22:19.534404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:22:19.534404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:22:19.534404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:19.534404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 512/1036] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:19.539890 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9063803-1a3a-4691-af38-88c18f1b0a13 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:19.544033 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:19.648263 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9063803-1a3a-4691-af38-88c18f1b0a13 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:19.682946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bece65f-9c0f-4b97-aada-3abeda03e7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:19.683559 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bece65f-9c0f-4b97-aada-3abeda03e7f0 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:19.683841 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bece65f-9c0f-4b97-aada-3abeda03e7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:19.684118 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bece65f-9c0f-4b97-aada-3abeda03e7f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:19.684753 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bece65f-9c0f-4b97-aada-3abeda03e7f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:19.685059 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bece65f-9c0f-4b97-aada-3abeda03e7f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:19.686445 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bece65f-9c0f-4b97-aada-3abeda03e7f0 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:19.698622 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 525/1037] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:22:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:19.703962 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f12e83-972f-4d62-b097-50e8002dc7b2 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:19.730407 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:19.807858 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f12e83-972f-4d62-b097-50e8002dc7b2 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:19.840999 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503ab4a-6b54-4e70-8b39-4752c8179ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:19.841629 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503ab4a-6b54-4e70-8b39-4752c8179ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:19.841910 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503ab4a-6b54-4e70-8b39-4752c8179ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:19.842181 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503ab4a-6b54-4e70-8b39-4752c8179ed7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:19.849795 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3503ab4a-6b54-4e70-8b39-4752c8179ed7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:19.851248 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3503ab4a-6b54-4e70-8b39-4752c8179ed7 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:19.863392 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-3503ab4a-6b54-4e70-8b39-4752c8179ed7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:22:19.863392 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:19.863392 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:19.863392 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:19.863392 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:19.863392 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:19.863392 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:19.863392 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:19.863392 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:19.863392 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:19.863392 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:19.863392 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:19.863392 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:22:19.863392 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:22:19.863392 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:22:19.863392 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:22:19.864343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:19.864343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:19.864343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:22:19.864343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:22:19.864343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:22:19.864343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:22:19.864343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:19.864343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:19.864343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:19.864343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:19.864343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:19.864343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:19.864343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:19.864343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:19.864343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:19.864343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:19.864343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:19.864343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:19.865282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:19.865282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:19.865282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:22:19.865282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:22:19.865282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:22:19.865282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:22:19.865282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:22:19.865282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:22:19.865282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:22:19.865282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:19.865282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 513/1038] 10.222.0.18 () {56 vars in 1093 bytes} [Thu Sep 1 16:22:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:19.872070 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bece65f-9c0f-4b97-aada-3abeda03e7f0 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:19.934092 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:19.973577 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bece65f-9c0f-4b97-aada-3abeda03e7f0 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:20.006889 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9780b7-d44e-4fd1-99fd-f9d7cf8deca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:20.007498 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9780b7-d44e-4fd1-99fd-f9d7cf8deca5 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:20.008295 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9780b7-d44e-4fd1-99fd-f9d7cf8deca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:20.008572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9780b7-d44e-4fd1-99fd-f9d7cf8deca5 None admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:20.009306 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9780b7-d44e-4fd1-99fd-f9d7cf8deca5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:20.009612 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9780b7-d44e-4fd1-99fd-f9d7cf8deca5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:20.011070 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9780b7-d44e-4fd1-99fd-f9d7cf8deca5 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:20.029030 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 526/1039] 10.222.0.18 () {56 vars in 1133 bytes} [Thu Sep 1 16:22:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:20.034485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503ab4a-6b54-4e70-8b39-4752c8179ed7 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:20.115956 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:20.135346 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503ab4a-6b54-4e70-8b39-4752c8179ed7 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:20.169432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab7ce2-364a-4dcd-b252-2a003372130c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:20.170102 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab7ce2-364a-4dcd-b252-2a003372130c None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:20.170436 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab7ce2-364a-4dcd-b252-2a003372130c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:20.170711 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab7ce2-364a-4dcd-b252-2a003372130c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:20.237273 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab7ce2-364a-4dcd-b252-2a003372130c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:20.238751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab7ce2-364a-4dcd-b252-2a003372130c None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:20.318549 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:20.338817 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 514/1040] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:22:20 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:22:20.344345 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9780b7-d44e-4fd1-99fd-f9d7cf8deca5 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:20.441339 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2f0ea-82fd-4c6e-89b0-e7763ec4ff72 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:20.441670 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2f0ea-82fd-4c6e-89b0-e7763ec4ff72 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:20.441899 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2f0ea-82fd-4c6e-89b0-e7763ec4ff72 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:20.443477 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 515/1041] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:22:20.448451 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df9d3d-b8cc-4308-9d9c-7f12bc583e59 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:20.448734 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df9d3d-b8cc-4308-9d9c-7f12bc583e59 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:20.449013 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df9d3d-b8cc-4308-9d9c-7f12bc583e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:20.454011 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9780b7-d44e-4fd1-99fd-f9d7cf8deca5 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:20.467364 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-88df9d3d-b8cc-4308-9d9c-7f12bc583e59 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:20.468579 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-88df9d3d-b8cc-4308-9d9c-7f12bc583e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:20.479033 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:20.494594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d01b6b-a112-43c3-9650-f8e0ad104da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:20.495228 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d01b6b-a112-43c3-9650-f8e0ad104da9 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:20.495507 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d01b6b-a112-43c3-9650-f8e0ad104da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:20.495776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d01b6b-a112-43c3-9650-f8e0ad104da9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:20.506699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d01b6b-a112-43c3-9650-f8e0ad104da9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:20.508700 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d01b6b-a112-43c3-9650-f8e0ad104da9 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:20.524583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-67d01b6b-a112-43c3-9650-f8e0ad104da9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:22:20.524583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:20.524583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:20.524583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:20.524583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:20.524583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:20.524583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:20.524583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:20.524583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:20.524583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:20.524583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:20.524583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:20.524583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:22:20.524583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:22:20.524583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:22:20.524583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:22:20.525553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:20.525553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:20.525553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:20.525553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:20.525553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:20.525553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:20.525553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:20.525553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:20.525553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:20.525553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:20.525553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:20.525553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:20.525553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:20.525553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:20.525553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:20.525553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:20.525553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:22:20.526492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:22:20.526492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:22:20.526492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:20.526492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 527/1042] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:20.531242 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d01b6b-a112-43c3-9650-f8e0ad104da9 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:20.545104 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 516/1043] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:20 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:22:20.628384 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d01b6b-a112-43c3-9650-f8e0ad104da9 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:20.661493 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749194b-fb42-4ab2-b2cf-f8fdffaf7823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:20.662149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749194b-fb42-4ab2-b2cf-f8fdffaf7823 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:20.662443 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749194b-fb42-4ab2-b2cf-f8fdffaf7823 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:20.662719 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749194b-fb42-4ab2-b2cf-f8fdffaf7823 None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:20.663350 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d749194b-fb42-4ab2-b2cf-f8fdffaf7823 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:20.663651 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d749194b-fb42-4ab2-b2cf-f8fdffaf7823 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:20.665577 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d749194b-fb42-4ab2-b2cf-f8fdffaf7823 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:20.677405 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 528/1044] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:20.683336 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df9d3d-b8cc-4308-9d9c-7f12bc583e59 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:20.782370 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df9d3d-b8cc-4308-9d9c-7f12bc583e59 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:20.795918 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:20.816894 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749194b-fb42-4ab2-b2cf-f8fdffaf7823 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:20.819797 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0e870-a11a-4840-b3f8-3730a4db24cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:20.820485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0e870-a11a-4840-b3f8-3730a4db24cf None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:20.820769 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0e870-a11a-4840-b3f8-3730a4db24cf None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:20.821048 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0e870-a11a-4840-b3f8-3730a4db24cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:20.828231 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d0e870-a11a-4840-b3f8-3730a4db24cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:20.829662 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d0e870-a11a-4840-b3f8-3730a4db24cf None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:20.840939 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-95d0e870-a11a-4840-b3f8-3730a4db24cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:22:20.840939 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:20.840939 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:20.840939 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:20.840939 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:20.840939 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:20.840939 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:20.840939 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:20.840939 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:20.840939 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:20.840939 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:20.840939 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:20.840939 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:22:20.840939 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:22:20.840939 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:22:20.841833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:22:20.841833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:20.841833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:20.841833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:20.841833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:20.841833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:20.841833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:20.841833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:20.841833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:20.841833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:20.841833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:20.841833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:20.841833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:20.841833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:20.841833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:20.841833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:20.841833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:20.842898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:22:20.842898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:22:20.842898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:22:20.842898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:22:20.842898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:22:20.842898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:22:20.842898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:22:20.842898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:20.842898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 517/1045] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:22:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:20.847691 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0e870-a11a-4840-b3f8-3730a4db24cf None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:20.859582 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:20.927959 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749194b-fb42-4ab2-b2cf-f8fdffaf7823 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:20.959590 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0e870-a11a-4840-b3f8-3730a4db24cf None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:20.968293 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70efdc6-846d-4885-87c5-3353cd7b9688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:20.969000 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70efdc6-846d-4885-87c5-3353cd7b9688 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:20.969283 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70efdc6-846d-4885-87c5-3353cd7b9688 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:20.969556 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70efdc6-846d-4885-87c5-3353cd7b9688 None admin] PATH_INFO: `/v3/services/image` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:20.970145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70efdc6-846d-4885-87c5-3353cd7b9688 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:20.971545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70efdc6-846d-4885-87c5-3353cd7b9688 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:20.980944 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-c70efdc6-846d-4885-87c5-3353cd7b9688 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:22:20.980944 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:20.980944 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:22:20.980944 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:22:20.980944 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:22:20.980944 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:22:20.980944 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:22:20.980944 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:22:20.980944 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:22:20.980944 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:20.980944 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:22:20.980944 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:20.980944 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:20.980944 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:20.980944 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:20.980944 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:20.980944 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:20.981927 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:20.981927 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:20.981927 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:20.981927 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:20.981927 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:20.981927 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:20.981927 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:22:20.981927 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:22:20.981927 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:22:20.981927 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:22:20.981927 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:20.981927 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:20.981927 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:20.981927 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:20.981927 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:20.981927 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:20.982795 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:20.982795 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:20.982795 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:20.982795 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:20.982795 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:20.982795 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:20.982795 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:20.982795 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:20.982795 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:20.982795 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:20.982795 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:22:20.982795 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:22:20.982795 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:22:20.982795 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:20.982795 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 529/1046] 10.222.0.18 () {56 vars in 1105 bytes} [Thu Sep 1 16:22:20 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:22:20.987949 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70efdc6-846d-4885-87c5-3353cd7b9688 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:20.998429 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51f22ea-89be-4eec-a45e-859b00dcbd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:20.999049 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f22ea-89be-4eec-a45e-859b00dcbd7b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:20.999330 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f22ea-89be-4eec-a45e-859b00dcbd7b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:20.999598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f22ea-89be-4eec-a45e-859b00dcbd7b None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:21.000287 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51f22ea-89be-4eec-a45e-859b00dcbd7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:21.000591 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51f22ea-89be-4eec-a45e-859b00dcbd7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:21.001996 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51f22ea-89be-4eec-a45e-859b00dcbd7b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:21.014936 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 518/1047] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:22:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:21.020627 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51f22ea-89be-4eec-a45e-859b00dcbd7b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:21.058415 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:21.093469 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:21.098983 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70efdc6-846d-4885-87c5-3353cd7b9688 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:21.136516 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51f22ea-89be-4eec-a45e-859b00dcbd7b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:21.139700 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7594c-6776-457a-afdc-af3b4eaa2571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:21.140302 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7594c-6776-457a-afdc-af3b4eaa2571 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:21.140582 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7594c-6776-457a-afdc-af3b4eaa2571 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:21.140856 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7594c-6776-457a-afdc-af3b4eaa2571 None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:21.141479 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b7594c-6776-457a-afdc-af3b4eaa2571 None admin] RBAC: Adding query filter params (name=image) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:21.141783 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b7594c-6776-457a-afdc-af3b4eaa2571 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:21.143183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b7594c-6776-457a-afdc-af3b4eaa2571 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:21.150278 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 530/1048] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:21.154455 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7594c-6776-457a-afdc-af3b4eaa2571 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:21.175553 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58daaf83-0c9e-4c37-bcba-ee09bec05f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:21.176166 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58daaf83-0c9e-4c37-bcba-ee09bec05f5e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:21.176444 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58daaf83-0c9e-4c37-bcba-ee09bec05f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:21.176714 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58daaf83-0c9e-4c37-bcba-ee09bec05f5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:21.177569 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58daaf83-0c9e-4c37-bcba-ee09bec05f5e None admin] RBAC: Adding query filter params (role.id=a889b69245b44da59d9020a0d39751cb, scope.project.id=36ce80c4e0d64562a9ab3961be76fad9, user.id=278e6d6aed414c758fe3e9f4df6bf81e) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:21.177856 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58daaf83-0c9e-4c37-bcba-ee09bec05f5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:21.179356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58daaf83-0c9e-4c37-bcba-ee09bec05f5e None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:21.185115 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 519/1049] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:22:21 2022] GET /identity/v3/role_assignments?user.id=278e6d6aed414c758fe3e9f4df6bf81e&scope.project.id=36ce80c4e0d64562a9ab3961be76fad9&role.id=a889b69245b44da59d9020a0d39751cb => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:21.272807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7594c-6776-457a-afdc-af3b4eaa2571 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:21.310331 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5f1e8-2761-4002-8b04-d0a4f33f41db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:21.310957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5f1e8-2761-4002-8b04-d0a4f33f41db None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:21.311237 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5f1e8-2761-4002-8b04-d0a4f33f41db None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:21.311506 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5f1e8-2761-4002-8b04-d0a4f33f41db None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:21.312149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b5f1e8-2761-4002-8b04-d0a4f33f41db None admin] RBAC: Adding query filter params (type=image) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:21.312451 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b5f1e8-2761-4002-8b04-d0a4f33f41db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:21.313843 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b5f1e8-2761-4002-8b04-d0a4f33f41db None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:21.319488 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 531/1050] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:21 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:21.325349 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5f1e8-2761-4002-8b04-d0a4f33f41db None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:21.325629 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fe01d-a965-4a44-aee9-5419ec5c1552 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:21.325905 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fe01d-a965-4a44-aee9-5419ec5c1552 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:21.326277 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fe01d-a965-4a44-aee9-5419ec5c1552 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:21.327639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 520/1051] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:21.332587 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092ec34-20b3-4b53-887a-8a8ccc3360a0 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:21.332841 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092ec34-20b3-4b53-887a-8a8ccc3360a0 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:21.333110 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092ec34-20b3-4b53-887a-8a8ccc3360a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:21.354579 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-8092ec34-20b3-4b53-887a-8a8ccc3360a0 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:21.355240 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:21.357543 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8092ec34-20b3-4b53-887a-8a8ccc3360a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:21.433778 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 521/1052] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:22:21.436249 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5f1e8-2761-4002-8b04-d0a4f33f41db None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:21.479359 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b701210-7693-4a72-9e08-6e59b1bd198f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:21.479975 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b701210-7693-4a72-9e08-6e59b1bd198f None admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:21.480255 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b701210-7693-4a72-9e08-6e59b1bd198f None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:21.480529 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b701210-7693-4a72-9e08-6e59b1bd198f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:21.481194 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b701210-7693-4a72-9e08-6e59b1bd198f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:21.482598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b701210-7693-4a72-9e08-6e59b1bd198f None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:21.510263 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 532/1053] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:22:21 2022] POST /identity/v3/endpoints => generated 327 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:22:21.700449 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092ec34-20b3-4b53-887a-8a8ccc3360a0 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:21.727886 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:21.801919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092ec34-20b3-4b53-887a-8a8ccc3360a0 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:21.841968 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26ae84-5262-482b-868b-484ddb353934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:21.842630 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26ae84-5262-482b-868b-484ddb353934 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:21.842908 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26ae84-5262-482b-868b-484ddb353934 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:21.843179 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26ae84-5262-482b-868b-484ddb353934 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:21.853092 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb26ae84-5262-482b-868b-484ddb353934 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:21.857761 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-cb26ae84-5262-482b-868b-484ddb353934 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:22:21.857761 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:21.857761 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:21.857761 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:21.857761 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:21.857761 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:21.857761 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:21.857761 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:21.857761 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:21.857761 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:21.857761 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:21.857761 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:21.857761 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:21.857761 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:21.857761 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:21.857761 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:21.858854 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:21.858854 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:21.858854 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:21.858854 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:21.858854 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:21.858854 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:21.858854 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:21.858854 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:21.858854 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:21.858854 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:21.858854 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:21.858854 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:21.858854 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:21.858854 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:21.858854 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:21.858854 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:21.859966 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:21.859966 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:21.859966 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:21.859966 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:21.859966 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:21.859966 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:21.859966 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:21.859966 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:21.859966 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:21.859966 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:21.859966 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:22:21.859966 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:21.860783 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 522/1054] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:22:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:21.865117 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b701210-7693-4a72-9e08-6e59b1bd198f None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:21.947104 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:21.967486 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b701210-7693-4a72-9e08-6e59b1bd198f None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:22.000616 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd127b4e-6e8c-4005-91a1-75ca6fe9377f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:22.001228 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd127b4e-6e8c-4005-91a1-75ca6fe9377f None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:22.001506 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd127b4e-6e8c-4005-91a1-75ca6fe9377f None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:22.001770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd127b4e-6e8c-4005-91a1-75ca6fe9377f None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:22.002452 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd127b4e-6e8c-4005-91a1-75ca6fe9377f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:22.002756 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd127b4e-6e8c-4005-91a1-75ca6fe9377f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:22.004111 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd127b4e-6e8c-4005-91a1-75ca6fe9377f None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:22.013217 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 533/1055] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:22:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:22.018736 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26ae84-5262-482b-868b-484ddb353934 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:22.099635 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:22.119391 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26ae84-5262-482b-868b-484ddb353934 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:22.154896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e59fee-4332-4956-b750-1e8565a9571b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:22.155531 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e59fee-4332-4956-b750-1e8565a9571b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:22.155808 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e59fee-4332-4956-b750-1e8565a9571b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:22.156074 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e59fee-4332-4956-b750-1e8565a9571b None admin] PATH_INFO: `/v3/users/183530e1521046ab902f37ae6cfac196` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:22.169998 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e59fee-4332-4956-b750-1e8565a9571b None admin] RBAC: Authorizing `identity:get_user(user_id=183530e1521046ab902f37ae6cfac196)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:22.171453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e59fee-4332-4956-b750-1e8565a9571b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:22.173325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 523/1056] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:22 2022] GET /identity/v3/users/183530e1521046ab902f37ae6cfac196 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:22.178946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd127b4e-6e8c-4005-91a1-75ca6fe9377f None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:22.278628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd127b4e-6e8c-4005-91a1-75ca6fe9377f None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:22.312171 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2f293-75f6-4509-86ce-3339bc6c6f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:22.312772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2f293-75f6-4509-86ce-3339bc6c6f8f None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:22.313055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2f293-75f6-4509-86ce-3339bc6c6f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:22.313327 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2f293-75f6-4509-86ce-3339bc6c6f8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:22.327756 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:22.389006 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2f293-75f6-4509-86ce-3339bc6c6f8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:22.390497 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2f293-75f6-4509-86ce-3339bc6c6f8f None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:22.495374 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 534/1057] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:22:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:22:22.496619 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e59fee-4332-4956-b750-1e8565a9571b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:22.599106 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e59fee-4332-4956-b750-1e8565a9571b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:22.633704 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9038f74-1fcd-47c6-9d44-15333aa4cfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:22.634494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9038f74-1fcd-47c6-9d44-15333aa4cfb0 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:22.634785 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9038f74-1fcd-47c6-9d44-15333aa4cfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:22.635068 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9038f74-1fcd-47c6-9d44-15333aa4cfb0 None admin] PATH_INFO: `/v3/projects/522bea8c53b24b43b44ab47d12c6d122` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:22.636197 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9038f74-1fcd-47c6-9d44-15333aa4cfb0 None admin] RBAC: Authorizing `identity:get_project(project_id=522bea8c53b24b43b44ab47d12c6d122)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:22.637635 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9038f74-1fcd-47c6-9d44-15333aa4cfb0 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:22.644170 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 524/1058] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:22 2022] GET /identity/v3/projects/522bea8c53b24b43b44ab47d12c6d122 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:22.645382 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2f293-75f6-4509-86ce-3339bc6c6f8f None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:22.692027 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:22.747374 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2f293-75f6-4509-86ce-3339bc6c6f8f None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:22.780551 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2e9e51-000a-489d-ac20-900ac12725a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:22.781165 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e9e51-000a-489d-ac20-900ac12725a5 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:22.781437 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e9e51-000a-489d-ac20-900ac12725a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:22.781706 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e9e51-000a-489d-ac20-900ac12725a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:22.782453 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2e9e51-000a-489d-ac20-900ac12725a5 None admin] RBAC: Adding query filter params (role.id=e8ba063905484d22993bcf04830091a3, scope.project.id=522bea8c53b24b43b44ab47d12c6d122, user.id=183530e1521046ab902f37ae6cfac196) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:22.782761 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2e9e51-000a-489d-ac20-900ac12725a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:22.784180 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2e9e51-000a-489d-ac20-900ac12725a5 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:22.790289 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 535/1059] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:22:22 2022] GET /identity/v3/role_assignments?user.id=183530e1521046ab902f37ae6cfac196&scope.project.id=522bea8c53b24b43b44ab47d12c6d122&role.id=e8ba063905484d22993bcf04830091a3 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:23.085218 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5f229-26bd-4826-8d32-29bafadb292e None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:23.085799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5f229-26bd-4826-8d32-29bafadb292e None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:23.086444 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5f229-26bd-4826-8d32-29bafadb292e None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:23.087740 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 525/1060] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:23.092860 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f58e9-400a-465d-8ee5-828372423f8c None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:23.093264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f58e9-400a-465d-8ee5-828372423f8c None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:23.093426 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f58e9-400a-465d-8ee5-828372423f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:23.111687 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-a38f58e9-400a-465d-8ee5-828372423f8c None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:23.112864 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-a38f58e9-400a-465d-8ee5-828372423f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:23.179218 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 536/1061] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:22:23.420698 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6263073-cf1d-49f0-900f-7b8511e02d42 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:23.421035 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6263073-cf1d-49f0-900f-7b8511e02d42 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:23.421259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6263073-cf1d-49f0-900f-7b8511e02d42 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:23.422748 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 526/1062] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:23.427573 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7bfd6-730a-45a8-b8a3-a23258676351 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:23.427865 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7bfd6-730a-45a8-b8a3-a23258676351 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:23.428136 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7bfd6-730a-45a8-b8a3-a23258676351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:23.446819 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6263073-cf1d-49f0-900f-7b8511e02d42 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:23.447076 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-66f7bfd6-730a-45a8-b8a3-a23258676351 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:23.448212 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-66f7bfd6-730a-45a8-b8a3-a23258676351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:23.458903 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:23.481172 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:23.526743 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 537/1063] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:22:23.558563 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6263073-cf1d-49f0-900f-7b8511e02d42 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:23.592367 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab81db4-95dc-40f1-bcc1-8454c09b042e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:23.592997 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab81db4-95dc-40f1-bcc1-8454c09b042e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:23.593278 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab81db4-95dc-40f1-bcc1-8454c09b042e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:23.593552 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab81db4-95dc-40f1-bcc1-8454c09b042e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:23.603671 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab81db4-95dc-40f1-bcc1-8454c09b042e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:23.608474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-dab81db4-95dc-40f1-bcc1-8454c09b042e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:22:23.608474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:23.608474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:23.608474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:23.608474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:23.608474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:23.608474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:23.608474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:23.608474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:23.608474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:23.608474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:23.608474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:23.608474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:23.608474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:23.608474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:23.608474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:23.609360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:23.609360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:23.609360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:23.609360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:23.609360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:23.609360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:23.609360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:23.609360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:23.609360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:23.609360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:23.609360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:23.609360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:23.609360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:23.609360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:23.609360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:23.609360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:23.611137 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:23.611137 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:23.611137 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:23.611137 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:23.611137 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:23.611137 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:23.611137 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:23.611137 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:23.611137 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:23.611137 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:23.611137 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:22:23.611137 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:23.611137 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 527/1064] 10.222.0.18 () {56 vars in 1120 bytes} [Thu Sep 1 16:22:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:22:23.615517 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7bfd6-730a-45a8-b8a3-a23258676351 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:23.713892 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7bfd6-730a-45a8-b8a3-a23258676351 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:23.752134 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b6156-1245-45fa-b2e5-29f25d0bddeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:23.752751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b6156-1245-45fa-b2e5-29f25d0bddeb None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:23.753035 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b6156-1245-45fa-b2e5-29f25d0bddeb None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:23.753311 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b6156-1245-45fa-b2e5-29f25d0bddeb None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:23.753979 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2b6156-1245-45fa-b2e5-29f25d0bddeb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:23.754311 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2b6156-1245-45fa-b2e5-29f25d0bddeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:23.755693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2b6156-1245-45fa-b2e5-29f25d0bddeb None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:23.768637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 538/1065] 10.222.0.18 () {56 vars in 1115 bytes} [Thu Sep 1 16:22:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:23.769788 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab81db4-95dc-40f1-bcc1-8454c09b042e None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:23.784813 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b6156-1245-45fa-b2e5-29f25d0bddeb None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:23.817314 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:23.846624 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:23.889810 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab81db4-95dc-40f1-bcc1-8454c09b042e None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:23.894088 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b6156-1245-45fa-b2e5-29f25d0bddeb None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:23.926477 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a820ef-f192-4635-8df8-a87db42f948f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:23.927114 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a820ef-f192-4635-8df8-a87db42f948f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:23.927397 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a820ef-f192-4635-8df8-a87db42f948f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:23.927673 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a820ef-f192-4635-8df8-a87db42f948f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:23.931675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6bc5d-02e6-40e1-bd65-334bae674721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:23.932285 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6bc5d-02e6-40e1-bd65-334bae674721 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:23.932567 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6bc5d-02e6-40e1-bd65-334bae674721 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:23.932846 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6bc5d-02e6-40e1-bd65-334bae674721 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:23.938622 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a820ef-f192-4635-8df8-a87db42f948f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:23.940059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a820ef-f192-4635-8df8-a87db42f948f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:23.956373 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-55a820ef-f192-4635-8df8-a87db42f948f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:22:23.956373 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:23.956373 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:23.956373 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:23.956373 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:23.956373 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:23.956373 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:23.956373 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:23.956373 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:23.956373 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:23.956373 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:23.956373 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:23.956373 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:22:23.956373 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:22:23.956373 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:22:23.956373 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:22:23.957303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:23.957303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:23.957303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:23.957303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:23.957303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:23.957303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:23.957303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:23.957303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:23.957303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:23.957303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:23.957303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:23.957303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:23.957303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:23.957303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:23.957303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:23.957303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:23.957303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:22:23.958388 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:22:23.958388 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:22:23.958388 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:23.958388 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 528/1066] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:22:23.963243 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a820ef-f192-4635-8df8-a87db42f948f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:24.004842 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c6bc5d-02e6-40e1-bd65-334bae674721 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:24.006269 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c6bc5d-02e6-40e1-bd65-334bae674721 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:24.037650 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:24.068505 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a820ef-f192-4635-8df8-a87db42f948f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:24.089559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:24.104876 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7786d-0d18-4786-bcc5-d2b0b9c91920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:24.105589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7786d-0d18-4786-bcc5-d2b0b9c91920 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:24.105765 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7786d-0d18-4786-bcc5-d2b0b9c91920 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:24.106035 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7786d-0d18-4786-bcc5-d2b0b9c91920 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:24.106700 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce7786d-0d18-4786-bcc5-d2b0b9c91920 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:24.107009 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce7786d-0d18-4786-bcc5-d2b0b9c91920 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:24.108343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce7786d-0d18-4786-bcc5-d2b0b9c91920 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:24.119170 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 539/1067] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:22:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 16:22:24.122296 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 529/1068] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:22:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:24.124793 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6bc5d-02e6-40e1-bd65-334bae674721 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:24.126212 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7786d-0d18-4786-bcc5-d2b0b9c91920 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:24.227886 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6bc5d-02e6-40e1-bd65-334bae674721 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:24.234571 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7786d-0d18-4786-bcc5-d2b0b9c91920 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:24.258452 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:24.265747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1a5db-1f76-4d41-a10c-7ca17b465b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:24.266425 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1a5db-1f76-4d41-a10c-7ca17b465b03 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:24.266714 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1a5db-1f76-4d41-a10c-7ca17b465b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:24.266994 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1a5db-1f76-4d41-a10c-7ca17b465b03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:24.271458 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfff6e8-b139-4cbf-8d4c-5ee2b0c2bd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:24.272095 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfff6e8-b139-4cbf-8d4c-5ee2b0c2bd6f None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:24.272382 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfff6e8-b139-4cbf-8d4c-5ee2b0c2bd6f None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:24.272661 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfff6e8-b139-4cbf-8d4c-5ee2b0c2bd6f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:24.278275 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb1a5db-1f76-4d41-a10c-7ca17b465b03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:24.279738 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb1a5db-1f76-4d41-a10c-7ca17b465b03 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:24.280956 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfff6e8-b139-4cbf-8d4c-5ee2b0c2bd6f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:24.282442 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfff6e8-b139-4cbf-8d4c-5ee2b0c2bd6f None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:24.294748 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-8dfff6e8-b139-4cbf-8d4c-5ee2b0c2bd6f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:22:24.294748 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:24.294748 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:24.294748 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:24.294748 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:24.294748 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:24.294748 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:24.294748 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:24.294748 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:24.294748 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:24.294748 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:24.294748 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:24.294748 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:22:24.294748 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:22:24.294748 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:22:24.294748 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:22:24.295685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:24.295685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:24.295685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:22:24.295685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:22:24.295685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:22:24.295685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:22:24.295685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:24.295685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:24.295685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:24.295685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:24.295685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:24.295685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:24.295685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:24.295685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:24.295685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:24.295685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:24.295685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:24.295685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:24.296607 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:24.296607 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:24.296607 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:22:24.296607 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:22:24.296607 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:22:24.296607 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:22:24.296607 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:22:24.296607 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:22:24.296607 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:22:24.296607 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:24.297124 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-8bb1a5db-1f76-4d41-a10c-7ca17b465b03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:22:24.297124 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:24.297124 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:24.297124 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:24.297124 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:24.297124 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:24.297124 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:24.297124 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:24.297124 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:24.297124 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:24.297124 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:24.297124 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:24.297124 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:22:24.297124 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:22:24.297124 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:22:24.297124 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:22:24.297951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:24.297951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:24.297951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:24.297951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:24.297951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:24.297951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:24.297951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:24.297951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:24.297951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:24.297951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:24.297951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:24.297951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:24.297951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:24.297951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:24.297951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:24.297951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:24.297951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:22:24.298864 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:22:24.298864 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:22:24.298864 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:24.299043 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 530/1069] 10.222.0.18 () {56 vars in 1093 bytes} [Thu Sep 1 16:22:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:22:24.301248 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 540/1070] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:22:24.301590 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfff6e8-b139-4cbf-8d4c-5ee2b0c2bd6f None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:24.307034 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1a5db-1f76-4d41-a10c-7ca17b465b03 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:24.353795 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:24.415405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfff6e8-b139-4cbf-8d4c-5ee2b0c2bd6f None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:24.425155 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1a5db-1f76-4d41-a10c-7ca17b465b03 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:24.451454 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24085b23-0ec1-4751-ac40-879697bd0068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:24.452065 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24085b23-0ec1-4751-ac40-879697bd0068 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:24.452344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24085b23-0ec1-4751-ac40-879697bd0068 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:24.452616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24085b23-0ec1-4751-ac40-879697bd0068 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:24.453332 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24085b23-0ec1-4751-ac40-879697bd0068 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:24.453635 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24085b23-0ec1-4751-ac40-879697bd0068 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:24.455128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24085b23-0ec1-4751-ac40-879697bd0068 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:24.462947 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6eece-aa07-42f3-aa69-b009ede898a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:24.463506 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6eece-aa07-42f3-aa69-b009ede898a5 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:24.463791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6eece-aa07-42f3-aa69-b009ede898a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:24.464089 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6eece-aa07-42f3-aa69-b009ede898a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:24.464815 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de6eece-aa07-42f3-aa69-b009ede898a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:24.465249 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de6eece-aa07-42f3-aa69-b009ede898a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:24.466641 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de6eece-aa07-42f3-aa69-b009ede898a5 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:24.474276 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 531/1071] 10.222.0.18 () {56 vars in 1133 bytes} [Thu Sep 1 16:22:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:24.479714 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24085b23-0ec1-4751-ac40-879697bd0068 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:24.490287 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 541/1072] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:22:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:24.595256 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:24.600468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24085b23-0ec1-4751-ac40-879697bd0068 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:24.640957 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ccd240-78f0-4d8d-a893-9178a90f78c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:24.641599 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccd240-78f0-4d8d-a893-9178a90f78c6 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:24.641885 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccd240-78f0-4d8d-a893-9178a90f78c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:24.642158 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccd240-78f0-4d8d-a893-9178a90f78c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:24.654188 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ccd240-78f0-4d8d-a893-9178a90f78c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:24.655750 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ccd240-78f0-4d8d-a893-9178a90f78c6 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:24.671197 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-03ccd240-78f0-4d8d-a893-9178a90f78c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:22:24.671197 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:24.671197 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:24.671197 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:24.671197 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:24.671197 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:24.671197 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:24.671197 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:24.671197 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:24.671197 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:24.671197 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:24.671197 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:24.671197 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:22:24.671197 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:22:24.671197 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:22:24.671197 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:22:24.672159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:24.672159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:24.672159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:24.672159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:24.672159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:24.672159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:24.672159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:24.672159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:24.672159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:24.672159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:24.672159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:24.672159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:24.672159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:24.672159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:24.672159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:24.672159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:24.672159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:22:24.673343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:22:24.673343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:22:24.673343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:24.673343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 532/1073] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:22:24.678469 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6eece-aa07-42f3-aa69-b009ede898a5 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:24.682428 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee74a52-5c42-4b7e-b6f5-a6ecaf5b0d89 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:24.682720 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee74a52-5c42-4b7e-b6f5-a6ecaf5b0d89 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:24.682998 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee74a52-5c42-4b7e-b6f5-a6ecaf5b0d89 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:24.684499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 533/1074] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:22:24.689331 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbdff4-f34b-4cb5-aeea-ead290ed5aa6 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:24.689624 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbdff4-f34b-4cb5-aeea-ead290ed5aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:24.689906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbdff4-f34b-4cb5-aeea-ead290ed5aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:24.708431 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-45dbdff4-f34b-4cb5-aeea-ead290ed5aa6 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:24.709625 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-45dbdff4-f34b-4cb5-aeea-ead290ed5aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:24.733886 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:24.786580 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 534/1075] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:22:24.799224 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6eece-aa07-42f3-aa69-b009ede898a5 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:24.833246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4119626-ed36-4910-ab80-7c20a1c38f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:24.833853 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4119626-ed36-4910-ab80-7c20a1c38f46 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:24.834137 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4119626-ed36-4910-ab80-7c20a1c38f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:24.834441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4119626-ed36-4910-ab80-7c20a1c38f46 None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:24.835085 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4119626-ed36-4910-ab80-7c20a1c38f46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:24.835390 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4119626-ed36-4910-ab80-7c20a1c38f46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:24.836801 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4119626-ed36-4910-ab80-7c20a1c38f46 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:24.847771 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 542/1076] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:22:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:24.853311 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dbdff4-f34b-4cb5-aeea-ead290ed5aa6 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:24.931227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:24.951366 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dbdff4-f34b-4cb5-aeea-ead290ed5aa6 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:24.983509 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1f669-757f-4e59-afc6-6fb89badd643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:24.984134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1f669-757f-4e59-afc6-6fb89badd643 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:24.984413 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1f669-757f-4e59-afc6-6fb89badd643 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:24.984685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1f669-757f-4e59-afc6-6fb89badd643 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:24.991121 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e1f669-757f-4e59-afc6-6fb89badd643 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:24.993026 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e1f669-757f-4e59-afc6-6fb89badd643 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:25.004149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-f7e1f669-757f-4e59-afc6-6fb89badd643 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:22:25.004149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:25.004149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:25.004149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:25.004149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:25.004149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:25.004149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:25.004149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:25.004149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:25.004149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:25.004149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:25.004149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:25.004149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:22:25.004149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:22:25.004149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:22:25.005183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:22:25.005183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:25.005183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:25.005183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:25.005183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:25.005183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:25.005183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:25.005183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:25.005183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:25.005183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:25.005183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:25.005183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:25.005183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:25.005183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:25.005183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:25.005183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:25.005183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:25.006316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:22:25.006316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:22:25.006316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:22:25.006316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:22:25.006316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:22:25.006316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:22:25.006316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:22:25.006316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:25.006316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 535/1077] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:22:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:25.011539 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4119626-ed36-4910-ab80-7c20a1c38f46 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:25.057513 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1f669-757f-4e59-afc6-6fb89badd643 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:25.111336 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4119626-ed36-4910-ab80-7c20a1c38f46 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:25.112919 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:25.148687 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6de255-67cd-4c8f-8edc-c92c2239669f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:25.149326 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6de255-67cd-4c8f-8edc-c92c2239669f None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:25.149589 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6de255-67cd-4c8f-8edc-c92c2239669f None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:25.149864 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6de255-67cd-4c8f-8edc-c92c2239669f None admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:25.150584 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6de255-67cd-4c8f-8edc-c92c2239669f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:25.150896 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6de255-67cd-4c8f-8edc-c92c2239669f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:25.152324 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6de255-67cd-4c8f-8edc-c92c2239669f None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:25.156683 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1f669-757f-4e59-afc6-6fb89badd643 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:25.164674 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 543/1078] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:22:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:25.170649 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6de255-67cd-4c8f-8edc-c92c2239669f None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:25.190613 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398a18c-ac4a-46dc-b5a1-5d7daf52b39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:25.191239 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398a18c-ac4a-46dc-b5a1-5d7daf52b39e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:25.191517 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398a18c-ac4a-46dc-b5a1-5d7daf52b39e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:25.191785 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398a18c-ac4a-46dc-b5a1-5d7daf52b39e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:25.199102 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:25.202701 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8398a18c-ac4a-46dc-b5a1-5d7daf52b39e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:25.207568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-8398a18c-ac4a-46dc-b5a1-5d7daf52b39e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:22:25.207568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:25.207568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:25.207568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:25.207568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:25.207568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:25.207568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:25.207568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:25.207568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:25.207568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:25.207568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:25.207568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:25.207568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:25.207568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:25.207568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:25.207568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:25.208485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:25.208485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:25.208485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:25.208485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:25.208485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:25.208485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:25.208485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:25.208485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:25.208485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:25.208485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:25.208485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:25.208485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:25.208485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:25.208485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:25.208485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:25.208485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:25.209319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:25.209319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:25.209319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:25.209319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:25.209319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:25.209319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:25.209319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:25.209319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:25.209319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:25.209319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:25.209319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:22:25.209319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:25.209319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 536/1079] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:22:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:22:25.214443 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398a18c-ac4a-46dc-b5a1-5d7daf52b39e None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:25.268166 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6de255-67cd-4c8f-8edc-c92c2239669f None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:25.300949 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96031e2a-ae89-43cf-9ed1-b17340d70581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:25.301601 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96031e2a-ae89-43cf-9ed1-b17340d70581 None admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:25.301884 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96031e2a-ae89-43cf-9ed1-b17340d70581 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:25.302161 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96031e2a-ae89-43cf-9ed1-b17340d70581 None admin] PATH_INFO: `/v3/projects/36ce80c4e0d64562a9ab3961be76fad9/users/278e6d6aed414c758fe3e9f4df6bf81e/roles/a889b69245b44da59d9020a0d39751cb` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:25.304416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96031e2a-ae89-43cf-9ed1-b17340d70581 None admin] RBAC: Authorizing `identity:create_grant(project_id=36ce80c4e0d64562a9ab3961be76fad9, user_id=278e6d6aed414c758fe3e9f4df6bf81e, role_id=a889b69245b44da59d9020a0d39751cb)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:25.306366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96031e2a-ae89-43cf-9ed1-b17340d70581 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:25.312151 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398a18c-ac4a-46dc-b5a1-5d7daf52b39e None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:25.319165 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 544/1080] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 1 16:22:25 2022] PUT /identity/v3/projects/36ce80c4e0d64562a9ab3961be76fad9/users/278e6d6aed414c758fe3e9f4df6bf81e/roles/a889b69245b44da59d9020a0d39751cb => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:22:25.347337 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ef2f1-79b8-44f8-9da8-161db3a3b5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:25.347979 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ef2f1-79b8-44f8-9da8-161db3a3b5bd None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:25.348263 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ef2f1-79b8-44f8-9da8-161db3a3b5bd None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:25.348580 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ef2f1-79b8-44f8-9da8-161db3a3b5bd None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:25.349226 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ef2f1-79b8-44f8-9da8-161db3a3b5bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:25.349536 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ef2f1-79b8-44f8-9da8-161db3a3b5bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:25.350980 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ef2f1-79b8-44f8-9da8-161db3a3b5bd None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:25.368574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 537/1081] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:22:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:25.370093 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96031e2a-ae89-43cf-9ed1-b17340d70581 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:25.396247 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:25.477229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96031e2a-ae89-43cf-9ed1-b17340d70581 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:25.515104 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3501451-15e1-4c58-83bb-0da778cb2170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:25.515732 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3501451-15e1-4c58-83bb-0da778cb2170 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:25.516012 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3501451-15e1-4c58-83bb-0da778cb2170 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:25.516284 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3501451-15e1-4c58-83bb-0da778cb2170 None admin] PATH_INFO: `/v3/users/183530e1521046ab902f37ae6cfac196` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:25.517405 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3501451-15e1-4c58-83bb-0da778cb2170 None admin] RBAC: Authorizing `identity:get_user(user_id=183530e1521046ab902f37ae6cfac196)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:25.518863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3501451-15e1-4c58-83bb-0da778cb2170 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:25.520430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 545/1082] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:25 2022] GET /identity/v3/users/183530e1521046ab902f37ae6cfac196 => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:25.526799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ef2f1-79b8-44f8-9da8-161db3a3b5bd None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:25.586778 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:25.625660 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ef2f1-79b8-44f8-9da8-161db3a3b5bd None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:25.659385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f00f37-3c78-417c-b60c-ada59d70af9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:25.660016 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f00f37-3c78-417c-b60c-ada59d70af9d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:25.660298 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f00f37-3c78-417c-b60c-ada59d70af9d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:25.660572 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f00f37-3c78-417c-b60c-ada59d70af9d None admin] PATH_INFO: `/v3/projects/522bea8c53b24b43b44ab47d12c6d122` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:25.661677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f00f37-3c78-417c-b60c-ada59d70af9d None admin] RBAC: Authorizing `identity:get_project(project_id=522bea8c53b24b43b44ab47d12c6d122)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:25.663198 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f00f37-3c78-417c-b60c-ada59d70af9d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:25.665429 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 538/1083] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:25 2022] GET /identity/v3/projects/522bea8c53b24b43b44ab47d12c6d122 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:25.671310 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3501451-15e1-4c58-83bb-0da778cb2170 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:25.758605 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:25.768538 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3501451-15e1-4c58-83bb-0da778cb2170 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:25.801580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788fb8b8-97c3-44f8-887a-bfb68a018fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:25.802248 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fb8b8-97c3-44f8-887a-bfb68a018fce None admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:25.802582 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fb8b8-97c3-44f8-887a-bfb68a018fce None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:25.802850 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fb8b8-97c3-44f8-887a-bfb68a018fce None admin] PATH_INFO: `/v3/projects/522bea8c53b24b43b44ab47d12c6d122/users/183530e1521046ab902f37ae6cfac196/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:25.804790 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788fb8b8-97c3-44f8-887a-bfb68a018fce None admin] RBAC: Authorizing `identity:create_grant(project_id=522bea8c53b24b43b44ab47d12c6d122, user_id=183530e1521046ab902f37ae6cfac196, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:25.806222 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788fb8b8-97c3-44f8-887a-bfb68a018fce None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:25.818994 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 546/1084] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 1 16:22:25 2022] PUT /identity/v3/projects/522bea8c53b24b43b44ab47d12c6d122/users/183530e1521046ab902f37ae6cfac196/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:22:27.175414 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd59a4f-8a86-459e-8573-63440b2c8dc5 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:27.175974 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd59a4f-8a86-459e-8573-63440b2c8dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:27.175974 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd59a4f-8a86-459e-8573-63440b2c8dc5 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:27.177425 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 539/1085] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:27.182366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b268456-212c-440e-b7cd-8eaf1b93fec1 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:27.182795 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b268456-212c-440e-b7cd-8eaf1b93fec1 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:27.182929 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b268456-212c-440e-b7cd-8eaf1b93fec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:27.200898 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-0b268456-212c-440e-b7cd-8eaf1b93fec1 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:27.202086 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-0b268456-212c-440e-b7cd-8eaf1b93fec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:27.267056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 547/1086] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:22:27.536746 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd59a4f-8a86-459e-8573-63440b2c8dc5 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:27.635931 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd59a4f-8a86-459e-8573-63440b2c8dc5 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:27.664346 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:27.670788 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac38ac-e467-48ff-bf76-1914f336e12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:27.671430 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac38ac-e467-48ff-bf76-1914f336e12b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:27.671715 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac38ac-e467-48ff-bf76-1914f336e12b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:27.671990 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac38ac-e467-48ff-bf76-1914f336e12b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:27.682299 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac38ac-e467-48ff-bf76-1914f336e12b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:27.687066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-15ac38ac-e467-48ff-bf76-1914f336e12b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:22:27.687066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:27.687066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:27.687066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:27.687066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:27.687066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:27.687066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:27.687066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:27.687066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:27.687066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:27.687066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:27.687066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:27.687066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:27.687066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:27.687066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:27.687066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:27.688184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:27.688184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:27.688184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:27.688184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:27.688184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:27.688184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:27.688184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:27.688184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:27.688184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:27.688184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:27.688184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:27.688184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:27.688184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:27.688184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:27.688184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:27.688184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:27.689047 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:27.689047 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:27.689047 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:27.689047 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:27.689047 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:27.689047 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:27.689047 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:27.689047 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:27.689047 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:27.689047 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:27.689047 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:22:27.689047 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:27.689047 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 540/1087] 10.222.0.18 () {56 vars in 1120 bytes} [Thu Sep 1 16:22:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:27.692747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75be1d-23bf-467d-addf-5cbee7fd0214 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:27.693049 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75be1d-23bf-467d-addf-5cbee7fd0214 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:27.693345 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75be1d-23bf-467d-addf-5cbee7fd0214 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:27.694325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac38ac-e467-48ff-bf76-1914f336e12b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:27.695368 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 548/1088] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:27.700229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a4e5d5-9d35-40a2-afde-5e545dbe7ce3 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:27.700522 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a4e5d5-9d35-40a2-afde-5e545dbe7ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:27.700800 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a4e5d5-9d35-40a2-afde-5e545dbe7ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:27.719059 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-08a4e5d5-9d35-40a2-afde-5e545dbe7ce3 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:27.720250 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-08a4e5d5-9d35-40a2-afde-5e545dbe7ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:27.787619 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 549/1089] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:22:27.793354 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac38ac-e467-48ff-bf76-1914f336e12b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:27.828271 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321bb47-1c86-4458-81ce-5368c7dd93de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:27.828876 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321bb47-1c86-4458-81ce-5368c7dd93de None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:27.829176 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321bb47-1c86-4458-81ce-5368c7dd93de None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:27.829432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321bb47-1c86-4458-81ce-5368c7dd93de None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:27.830087 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b321bb47-1c86-4458-81ce-5368c7dd93de None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:27.830429 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b321bb47-1c86-4458-81ce-5368c7dd93de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:27.831775 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b321bb47-1c86-4458-81ce-5368c7dd93de None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:27.839704 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 541/1090] 10.222.0.18 () {56 vars in 1115 bytes} [Thu Sep 1 16:22:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:27.845334 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a4e5d5-9d35-40a2-afde-5e545dbe7ce3 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:27.857166 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:27.947417 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a4e5d5-9d35-40a2-afde-5e545dbe7ce3 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:27.979747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37378d-4707-4ca8-9d62-c93437038d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:27.980372 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37378d-4707-4ca8-9d62-c93437038d72 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:27.980661 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37378d-4707-4ca8-9d62-c93437038d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:27.980935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37378d-4707-4ca8-9d62-c93437038d72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:27.990632 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37378d-4707-4ca8-9d62-c93437038d72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:27.992613 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37378d-4707-4ca8-9d62-c93437038d72 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:28.006946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-7c37378d-4707-4ca8-9d62-c93437038d72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:22:28.006946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:28.006946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:28.006946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:28.006946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:28.006946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:28.006946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:28.006946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:28.006946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:28.006946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:28.006946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:28.006946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:28.006946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:22:28.006946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:22:28.006946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:22:28.006946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:22:28.008416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:28.008416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:28.008416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:28.008416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:28.008416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:28.008416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:28.008416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:28.008416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:28.008416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:28.008416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:28.008416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:28.008416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:28.008416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:28.008416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:28.008416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:28.008416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:28.008416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:22:28.009324 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:22:28.009324 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:22:28.009324 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:28.009324 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 550/1091] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:22:28.014473 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321bb47-1c86-4458-81ce-5368c7dd93de None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:28.066357 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:28.066520 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37378d-4707-4ca8-9d62-c93437038d72 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:28.113436 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321bb47-1c86-4458-81ce-5368c7dd93de None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:28.127916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:28.148120 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdee4aa-fdb5-4767-9012-5817003e8834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:28.148733 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdee4aa-fdb5-4767-9012-5817003e8834 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:28.149016 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdee4aa-fdb5-4767-9012-5817003e8834 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:28.149289 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdee4aa-fdb5-4767-9012-5817003e8834 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:28.149925 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdee4aa-fdb5-4767-9012-5817003e8834 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:28.150230 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdee4aa-fdb5-4767-9012-5817003e8834 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:28.151604 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdee4aa-fdb5-4767-9012-5817003e8834 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:28.163063 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 542/1092] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:22:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:28.169107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdee4aa-fdb5-4767-9012-5817003e8834 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:28.171139 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37378d-4707-4ca8-9d62-c93437038d72 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:28.204608 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46caa517-26bb-4406-bb86-f0ad9c8b1148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:28.205230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46caa517-26bb-4406-bb86-f0ad9c8b1148 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:28.205507 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46caa517-26bb-4406-bb86-f0ad9c8b1148 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:28.205782 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46caa517-26bb-4406-bb86-f0ad9c8b1148 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:28.216351 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46caa517-26bb-4406-bb86-f0ad9c8b1148 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:28.221532 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-46caa517-26bb-4406-bb86-f0ad9c8b1148 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:22:28.221532 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:28.221532 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:28.221532 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:28.221532 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:28.221532 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:28.221532 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:28.221532 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:28.221532 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:28.221532 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:28.221532 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:28.221532 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:28.221532 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:28.221532 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:28.221532 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:28.221532 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:28.222692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:28.222692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:28.222692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:28.222692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:28.222692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:28.222692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:28.222692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:28.222692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:28.222692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:28.222692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:28.222692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:28.222692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:28.222692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:28.222692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:28.222692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:28.222692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:28.223988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:28.223988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:28.223988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:28.223988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:28.223988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:28.223988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:28.223988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:28.223988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:28.223988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:28.223988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:28.223988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:22:28.223988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:28.223988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 551/1093] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:22:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:22:28.229016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46caa517-26bb-4406-bb86-f0ad9c8b1148 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:28.272280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdee4aa-fdb5-4767-9012-5817003e8834 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:28.287210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:28.307707 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d2ccfe-4fb3-4d03-917a-2aa4a0b33755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:28.308332 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d2ccfe-4fb3-4d03-917a-2aa4a0b33755 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:28.308627 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d2ccfe-4fb3-4d03-917a-2aa4a0b33755 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:28.308884 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d2ccfe-4fb3-4d03-917a-2aa4a0b33755 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:28.315773 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d2ccfe-4fb3-4d03-917a-2aa4a0b33755 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:28.317263 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d2ccfe-4fb3-4d03-917a-2aa4a0b33755 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:28.329168 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46caa517-26bb-4406-bb86-f0ad9c8b1148 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:28.329356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-29d2ccfe-4fb3-4d03-917a-2aa4a0b33755 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:22:28.329356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:28.329356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:28.329356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:28.329356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:28.329356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:28.329356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:28.329356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:28.329356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:28.329356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:28.329356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:28.329356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:28.329356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:22:28.329356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:22:28.329356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:22:28.329356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:22:28.330483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:28.330483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:28.330483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:22:28.330483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:22:28.330483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:22:28.330483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:22:28.330483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:28.330483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:28.330483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:28.330483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:28.330483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:28.330483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:28.330483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:28.330483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:28.330483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:28.330483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:28.330483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:28.330483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:28.331416 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:28.331416 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:28.331416 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:22:28.331416 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:22:28.331416 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:22:28.331416 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:22:28.331416 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:22:28.331416 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:22:28.331416 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:22:28.331416 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:28.334381 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 543/1094] 10.222.0.18 () {56 vars in 1093 bytes} [Thu Sep 1 16:22:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:22:28.336636 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d2ccfe-4fb3-4d03-917a-2aa4a0b33755 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:28.352546 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:28.364923 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf3de9-12b9-4fd2-8c46-92922decfffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:28.365643 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf3de9-12b9-4fd2-8c46-92922decfffb None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:28.365926 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf3de9-12b9-4fd2-8c46-92922decfffb None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:28.366420 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf3de9-12b9-4fd2-8c46-92922decfffb None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:28.367091 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebf3de9-12b9-4fd2-8c46-92922decfffb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:28.367877 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebf3de9-12b9-4fd2-8c46-92922decfffb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:28.368972 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebf3de9-12b9-4fd2-8c46-92922decfffb None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:28.377845 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 552/1095] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:22:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:28.383751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf3de9-12b9-4fd2-8c46-92922decfffb None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:28.437863 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d2ccfe-4fb3-4d03-917a-2aa4a0b33755 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:28.471832 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5d77b-18b5-4533-a7b9-defa5e18b4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:28.472441 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5d77b-18b5-4533-a7b9-defa5e18b4dd None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:28.472732 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5d77b-18b5-4533-a7b9-defa5e18b4dd None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:28.473007 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5d77b-18b5-4533-a7b9-defa5e18b4dd None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:28.473741 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b5d77b-18b5-4533-a7b9-defa5e18b4dd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:28.474050 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b5d77b-18b5-4533-a7b9-defa5e18b4dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:28.475506 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b5d77b-18b5-4533-a7b9-defa5e18b4dd None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:28.481978 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf3de9-12b9-4fd2-8c46-92922decfffb None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:28.493472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 544/1096] 10.222.0.18 () {56 vars in 1133 bytes} [Thu Sep 1 16:22:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:28.499631 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5d77b-18b5-4533-a7b9-defa5e18b4dd None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:28.516685 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff442f6f-f2ce-4bb9-b59a-dbbeb1731334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:28.517314 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff442f6f-f2ce-4bb9-b59a-dbbeb1731334 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:28.517601 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff442f6f-f2ce-4bb9-b59a-dbbeb1731334 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:28.517883 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff442f6f-f2ce-4bb9-b59a-dbbeb1731334 None admin] PATH_INFO: `/v3/users/183530e1521046ab902f37ae6cfac196` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:28.519042 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff442f6f-f2ce-4bb9-b59a-dbbeb1731334 None admin] RBAC: Authorizing `identity:get_user(user_id=183530e1521046ab902f37ae6cfac196)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:28.520485 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff442f6f-f2ce-4bb9-b59a-dbbeb1731334 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:28.522089 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 553/1097] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:28 2022] GET /identity/v3/users/183530e1521046ab902f37ae6cfac196 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:28.527689 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff442f6f-f2ce-4bb9-b59a-dbbeb1731334 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:28.528787 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:28.582810 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:28.599105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5d77b-18b5-4533-a7b9-defa5e18b4dd None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:28.629991 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff442f6f-f2ce-4bb9-b59a-dbbeb1731334 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:28.632567 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70aa251-ca1c-48ca-a66f-00c48d70e782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:28.633716 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70aa251-ca1c-48ca-a66f-00c48d70e782 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:28.633992 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70aa251-ca1c-48ca-a66f-00c48d70e782 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:28.634282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70aa251-ca1c-48ca-a66f-00c48d70e782 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:28.663031 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464867a5-3c9a-4417-829b-3ac45526a3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:28.663660 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464867a5-3c9a-4417-829b-3ac45526a3b0 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:28.663946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464867a5-3c9a-4417-829b-3ac45526a3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:28.664223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464867a5-3c9a-4417-829b-3ac45526a3b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:28.700195 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70aa251-ca1c-48ca-a66f-00c48d70e782 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:28.701619 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70aa251-ca1c-48ca-a66f-00c48d70e782 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:28.730719 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:28.730977 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464867a5-3c9a-4417-829b-3ac45526a3b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:28.732429 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464867a5-3c9a-4417-829b-3ac45526a3b0 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:28.782568 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:28.800706 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 545/1098] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:22:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 16:22:28.806947 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70aa251-ca1c-48ca-a66f-00c48d70e782 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:28.829234 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 554/1099] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:22:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:22:28.834917 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464867a5-3c9a-4417-829b-3ac45526a3b0 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:28.903029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70aa251-ca1c-48ca-a66f-00c48d70e782 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:28.931150 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464867a5-3c9a-4417-829b-3ac45526a3b0 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:28.931303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:28.937362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea2d94-534a-49e4-8521-b247a567ba79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:28.937988 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea2d94-534a-49e4-8521-b247a567ba79 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:28.938335 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea2d94-534a-49e4-8521-b247a567ba79 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:28.938618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea2d94-534a-49e4-8521-b247a567ba79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:28.948476 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea2d94-534a-49e4-8521-b247a567ba79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:28.949894 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea2d94-534a-49e4-8521-b247a567ba79 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:28.963567 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1fd2b-8498-4367-ac96-990528d41730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:28.964187 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1fd2b-8498-4367-ac96-990528d41730 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:28.964470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1fd2b-8498-4367-ac96-990528d41730 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:28.964745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1fd2b-8498-4367-ac96-990528d41730 None admin] PATH_INFO: `/v3/projects/522bea8c53b24b43b44ab47d12c6d122` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:28.965061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-d9ea2d94-534a-49e4-8521-b247a567ba79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:22:28.965061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:28.965061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:28.965061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:28.965061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:28.965061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:28.965061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:28.965061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:28.965061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:28.965061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:28.965061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:28.965061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:28.965061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:22:28.965061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:22:28.965061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:22:28.965061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:22:28.965928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:28.965928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:28.965928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:28.965928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:28.965928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:28.965928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:28.965928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:28.965928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:28.965928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:28.965928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:28.965928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:28.965928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:28.965928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:28.965928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:28.965928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:28.965928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:28.965928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:22:28.966847 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:22:28.966847 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:22:28.966847 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:28.967016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1fd2b-8498-4367-ac96-990528d41730 None admin] RBAC: Authorizing `identity:get_project(project_id=522bea8c53b24b43b44ab47d12c6d122)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:28.967084 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 546/1100] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:22:28.967382 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1fd2b-8498-4367-ac96-990528d41730 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:28.973832 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 555/1101] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:28 2022] GET /identity/v3/projects/522bea8c53b24b43b44ab47d12c6d122 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:28.973976 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea2d94-534a-49e4-8521-b247a567ba79 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:28.974843 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1fd2b-8498-4367-ac96-990528d41730 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:28.997237 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:29.069895 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea2d94-534a-49e4-8521-b247a567ba79 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:29.074117 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1fd2b-8498-4367-ac96-990528d41730 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:29.103988 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd4d5ea-7a74-4e27-bd46-0d09bb8f18c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:29.104526 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4d5ea-7a74-4e27-bd46-0d09bb8f18c7 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:29.104803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4d5ea-7a74-4e27-bd46-0d09bb8f18c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:29.105071 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4d5ea-7a74-4e27-bd46-0d09bb8f18c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:29.105701 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd4d5ea-7a74-4e27-bd46-0d09bb8f18c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:29.106002 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd4d5ea-7a74-4e27-bd46-0d09bb8f18c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:29.106548 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d1c0b0-094c-4cb5-9758-24fb86fce4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:29.107141 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1c0b0-094c-4cb5-9758-24fb86fce4c8 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:29.107432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd4d5ea-7a74-4e27-bd46-0d09bb8f18c7 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:29.107522 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1c0b0-094c-4cb5-9758-24fb86fce4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:29.107721 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1c0b0-094c-4cb5-9758-24fb86fce4c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:29.108413 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d1c0b0-094c-4cb5-9758-24fb86fce4c8 None admin] RBAC: Adding query filter params (role.id=e8ba063905484d22993bcf04830091a3, scope.project.id=522bea8c53b24b43b44ab47d12c6d122, user.id=183530e1521046ab902f37ae6cfac196) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:29.108725 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d1c0b0-094c-4cb5-9758-24fb86fce4c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:29.110128 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d1c0b0-094c-4cb5-9758-24fb86fce4c8 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:29.115782 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 556/1102] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:22:28 2022] GET /identity/v3/role_assignments?user.id=183530e1521046ab902f37ae6cfac196&scope.project.id=522bea8c53b24b43b44ab47d12c6d122&role.id=e8ba063905484d22993bcf04830091a3 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:29.120139 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 547/1103] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:22:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:29.125747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d1c0b0-094c-4cb5-9758-24fb86fce4c8 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:29.216320 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:29.232965 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d1c0b0-094c-4cb5-9758-24fb86fce4c8 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:29.268251 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688fe416-ef12-4c72-8a7d-763aa881e64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:29.268882 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fe416-ef12-4c72-8a7d-763aa881e64a None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:29.269160 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fe416-ef12-4c72-8a7d-763aa881e64a None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:29.269435 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fe416-ef12-4c72-8a7d-763aa881e64a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:29.276688 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688fe416-ef12-4c72-8a7d-763aa881e64a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:29.278120 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688fe416-ef12-4c72-8a7d-763aa881e64a None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:29.289745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-688fe416-ef12-4c72-8a7d-763aa881e64a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:22:29.289745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:29.289745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:29.289745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:29.289745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:29.289745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:29.289745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:29.289745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:29.289745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:29.289745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:29.289745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:29.289745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:29.289745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:22:29.289745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:22:29.289745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:22:29.290728 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:22:29.290728 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:29.290728 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:29.290728 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:29.290728 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:29.290728 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:29.290728 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:29.290728 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:29.290728 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:29.290728 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:29.290728 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:29.290728 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:29.290728 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:29.290728 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:29.290728 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:29.290728 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:29.290728 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:29.291639 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:22:29.291639 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:22:29.291639 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:22:29.291639 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:22:29.291639 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:22:29.291639 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:22:29.291639 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:22:29.291639 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:29.291639 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 557/1104] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:22:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:29.296903 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd4d5ea-7a74-4e27-bd46-0d09bb8f18c7 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:29.340931 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:29.401116 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd4d5ea-7a74-4e27-bd46-0d09bb8f18c7 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:29.433782 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b185b9-2849-422b-95a3-861a77b0399d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:29.434487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b185b9-2849-422b-95a3-861a77b0399d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:29.434768 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b185b9-2849-422b-95a3-861a77b0399d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:29.435038 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b185b9-2849-422b-95a3-861a77b0399d None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:29.435725 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b185b9-2849-422b-95a3-861a77b0399d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:29.436029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b185b9-2849-422b-95a3-861a77b0399d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:29.437439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b185b9-2849-422b-95a3-861a77b0399d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:29.448643 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 548/1105] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:22:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:29.454861 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688fe416-ef12-4c72-8a7d-763aa881e64a None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:29.552644 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688fe416-ef12-4c72-8a7d-763aa881e64a None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:29.584158 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29291d12-4aea-46a2-b0a2-a1ebe86fc543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:29.584760 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29291d12-4aea-46a2-b0a2-a1ebe86fc543 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:29.585036 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29291d12-4aea-46a2-b0a2-a1ebe86fc543 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:29.585304 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29291d12-4aea-46a2-b0a2-a1ebe86fc543 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:29.586568 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29291d12-4aea-46a2-b0a2-a1ebe86fc543 None admin] RBAC: Adding query filter params (role.id=a889b69245b44da59d9020a0d39751cb, scope.project.id=36ce80c4e0d64562a9ab3961be76fad9, user.id=278e6d6aed414c758fe3e9f4df6bf81e) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:29.586873 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29291d12-4aea-46a2-b0a2-a1ebe86fc543 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:29.588295 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29291d12-4aea-46a2-b0a2-a1ebe86fc543 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:29.593958 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 558/1106] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:22:29 2022] GET /identity/v3/role_assignments?user.id=278e6d6aed414c758fe3e9f4df6bf81e&scope.project.id=36ce80c4e0d64562a9ab3961be76fad9&role.id=a889b69245b44da59d9020a0d39751cb => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:31.025288 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e7687-8c98-4bfb-b2a9-570eb316e39b None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:31.025819 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e7687-8c98-4bfb-b2a9-570eb316e39b None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:31.025819 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e7687-8c98-4bfb-b2a9-570eb316e39b None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:31.027205 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 549/1107] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:31.032165 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9453e2-d6e1-47d4-8d72-e729ee404fec None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:31.032562 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9453e2-d6e1-47d4-8d72-e729ee404fec None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:31.032720 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9453e2-d6e1-47d4-8d72-e729ee404fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:31.051010 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-0e9453e2-d6e1-47d4-8d72-e729ee404fec None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:31.052175 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-0e9453e2-d6e1-47d4-8d72-e729ee404fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:31.054111 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:31.116822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 559/1108] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:22:31.385813 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e7687-8c98-4bfb-b2a9-570eb316e39b None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:31.484602 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e7687-8c98-4bfb-b2a9-570eb316e39b None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:31.486074 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:31.519728 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511792a7-83dd-4011-b79b-9a0298c0b3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:31.520355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511792a7-83dd-4011-b79b-9a0298c0b3d5 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:31.520642 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511792a7-83dd-4011-b79b-9a0298c0b3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:31.520921 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511792a7-83dd-4011-b79b-9a0298c0b3d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:31.522034 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511792a7-83dd-4011-b79b-9a0298c0b3d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:31.523630 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511792a7-83dd-4011-b79b-9a0298c0b3d5 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:31.525196 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 550/1109] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:31.531008 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9453e2-d6e1-47d4-8d72-e729ee404fec None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:31.626068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9453e2-d6e1-47d4-8d72-e729ee404fec None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:31.658123 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140dc35-676c-49c0-be56-006b378ae724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:31.658743 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140dc35-676c-49c0-be56-006b378ae724 None admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:31.659022 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140dc35-676c-49c0-be56-006b378ae724 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:31.659293 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140dc35-676c-49c0-be56-006b378ae724 None admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:31.659927 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140dc35-676c-49c0-be56-006b378ae724 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:31.661342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140dc35-676c-49c0-be56-006b378ae724 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:31.671023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:31.682805 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 560/1110] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 1 16:22:31 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:22:33.579479 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa2d4c-6941-40b3-8360-dc6c34bd7db7 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:33.579999 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa2d4c-6941-40b3-8360-dc6c34bd7db7 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:33.580085 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa2d4c-6941-40b3-8360-dc6c34bd7db7 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:33.581367 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 551/1111] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:33.586940 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690fb0a-5600-4f89-9c41-cf7f4aa6fa20 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:33.587383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690fb0a-5600-4f89-9c41-cf7f4aa6fa20 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:33.587477 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690fb0a-5600-4f89-9c41-cf7f4aa6fa20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:33.605128 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-4690fb0a-5600-4f89-9c41-cf7f4aa6fa20 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:33.606322 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-4690fb0a-5600-4f89-9c41-cf7f4aa6fa20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:33.670655 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 561/1112] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:22:33.941747 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa2d4c-6941-40b3-8360-dc6c34bd7db7 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:34.040334 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa2d4c-6941-40b3-8360-dc6c34bd7db7 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:34.073116 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213cde2-726e-477e-9a70-62cddaa15973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:34.073729 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213cde2-726e-477e-9a70-62cddaa15973 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:34.074007 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213cde2-726e-477e-9a70-62cddaa15973 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:34.074308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213cde2-726e-477e-9a70-62cddaa15973 None admin] PATH_INFO: `/v3/roles/f2a6e936aa294323ba7087bf377276b2` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:34.075676 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213cde2-726e-477e-9a70-62cddaa15973 None admin] RBAC: Authorizing `identity:get_role(role_id=f2a6e936aa294323ba7087bf377276b2)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:34.077017 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213cde2-726e-477e-9a70-62cddaa15973 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:34.078498 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 552/1113] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:33 2022] GET /identity/v3/roles/f2a6e936aa294323ba7087bf377276b2 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:34.083900 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690fb0a-5600-4f89-9c41-cf7f4aa6fa20 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:34.178952 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690fb0a-5600-4f89-9c41-cf7f4aa6fa20 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:34.201424 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:34.212222 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b078da-065e-40cc-8176-e3c873991db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:34.212839 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b078da-065e-40cc-8176-e3c873991db2 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:34.213120 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b078da-065e-40cc-8176-e3c873991db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:34.213399 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b078da-065e-40cc-8176-e3c873991db2 None admin] PATH_INFO: `/v3/users/c238e67923344fce8c56e30a98100e92` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:34.226511 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b078da-065e-40cc-8176-e3c873991db2 None admin] RBAC: Authorizing `identity:get_user(user_id=c238e67923344fce8c56e30a98100e92)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:34.227938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b078da-065e-40cc-8176-e3c873991db2 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:34.229509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 562/1114] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:34 2022] GET /identity/v3/users/c238e67923344fce8c56e30a98100e92 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:34.235136 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213cde2-726e-477e-9a70-62cddaa15973 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:34.274429 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:34.333338 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213cde2-726e-477e-9a70-62cddaa15973 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:34.366914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a9014-3f0e-4397-a72a-f718ad8690f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:34.367532 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a9014-3f0e-4397-a72a-f718ad8690f2 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:34.368163 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a9014-3f0e-4397-a72a-f718ad8690f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:34.368163 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a9014-3f0e-4397-a72a-f718ad8690f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:34.434916 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a9014-3f0e-4397-a72a-f718ad8690f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:34.436347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a9014-3f0e-4397-a72a-f718ad8690f2 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:34.472518 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:34.532432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 553/1115] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:22:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:22:34.538896 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b078da-065e-40cc-8176-e3c873991db2 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:34.633556 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b078da-065e-40cc-8176-e3c873991db2 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:34.667070 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ca4fd-5446-4caf-9809-05f5c6c363a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:34.667684 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ca4fd-5446-4caf-9809-05f5c6c363a5 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:34.667967 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ca4fd-5446-4caf-9809-05f5c6c363a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:34.668242 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ca4fd-5446-4caf-9809-05f5c6c363a5 None admin] PATH_INFO: `/v3/projects/522bea8c53b24b43b44ab47d12c6d122` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:34.669347 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ca4fd-5446-4caf-9809-05f5c6c363a5 None admin] RBAC: Authorizing `identity:get_project(project_id=522bea8c53b24b43b44ab47d12c6d122)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:34.670795 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ca4fd-5446-4caf-9809-05f5c6c363a5 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:34.672369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 563/1116] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:34 2022] GET /identity/v3/projects/522bea8c53b24b43b44ab47d12c6d122 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:34.678425 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a9014-3f0e-4397-a72a-f718ad8690f2 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:34.774023 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a9014-3f0e-4397-a72a-f718ad8690f2 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:34.806550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c29b86-afb1-4150-8aed-a0c49b53eea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:34.807138 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c29b86-afb1-4150-8aed-a0c49b53eea1 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:34.807418 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c29b86-afb1-4150-8aed-a0c49b53eea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:34.807689 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c29b86-afb1-4150-8aed-a0c49b53eea1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:34.808401 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c29b86-afb1-4150-8aed-a0c49b53eea1 None admin] RBAC: Adding query filter params (role.id=f2a6e936aa294323ba7087bf377276b2, scope.project.id=522bea8c53b24b43b44ab47d12c6d122, user.id=c238e67923344fce8c56e30a98100e92) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:34.808710 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c29b86-afb1-4150-8aed-a0c49b53eea1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:34.810105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c29b86-afb1-4150-8aed-a0c49b53eea1 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:34.815277 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 554/1117] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:22:34 2022] GET /identity/v3/role_assignments?user.id=c238e67923344fce8c56e30a98100e92&scope.project.id=522bea8c53b24b43b44ab47d12c6d122&role.id=f2a6e936aa294323ba7087bf377276b2 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:36.648988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd122f-cc27-4df8-b5ed-2945a48ac697 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:36.649486 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd122f-cc27-4df8-b5ed-2945a48ac697 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:36.649486 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd122f-cc27-4df8-b5ed-2945a48ac697 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:36.650805 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 564/1118] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:36.655816 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7e94f-a032-41fb-a241-342da3c6640d None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:36.656233 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7e94f-a032-41fb-a241-342da3c6640d None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:36.656366 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7e94f-a032-41fb-a241-342da3c6640d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:36.662665 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:36.675743 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-56f7e94f-a032-41fb-a241-342da3c6640d None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:36.676918 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-56f7e94f-a032-41fb-a241-342da3c6640d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:36.741135 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 555/1119] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:22:37.008423 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd122f-cc27-4df8-b5ed-2945a48ac697 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:37.058298 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:37.107037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd122f-cc27-4df8-b5ed-2945a48ac697 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:37.139372 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa063b0-dccf-4794-8f88-dc6a5e50abef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:37.139985 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa063b0-dccf-4794-8f88-dc6a5e50abef None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:37.140269 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa063b0-dccf-4794-8f88-dc6a5e50abef None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:37.140543 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa063b0-dccf-4794-8f88-dc6a5e50abef None admin] PATH_INFO: `/v3/roles/f2a6e936aa294323ba7087bf377276b2` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:37.141898 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa063b0-dccf-4794-8f88-dc6a5e50abef None admin] RBAC: Authorizing `identity:get_role(role_id=f2a6e936aa294323ba7087bf377276b2)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:37.143281 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa063b0-dccf-4794-8f88-dc6a5e50abef None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:37.144564 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 565/1120] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:37 2022] GET /identity/v3/roles/f2a6e936aa294323ba7087bf377276b2 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:37.150777 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f7e94f-a032-41fb-a241-342da3c6640d None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:37.246638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f7e94f-a032-41fb-a241-342da3c6640d None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:37.275436 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:37.280510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0470f2a2-5744-463a-9136-844ccb744c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:37.281134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470f2a2-5744-463a-9136-844ccb744c12 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:37.281419 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470f2a2-5744-463a-9136-844ccb744c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:37.281697 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470f2a2-5744-463a-9136-844ccb744c12 None admin] PATH_INFO: `/v3/users/c238e67923344fce8c56e30a98100e92` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:37.282831 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0470f2a2-5744-463a-9136-844ccb744c12 None admin] RBAC: Authorizing `identity:get_user(user_id=c238e67923344fce8c56e30a98100e92)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:37.284252 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0470f2a2-5744-463a-9136-844ccb744c12 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:37.285823 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 556/1121] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:37 2022] GET /identity/v3/users/c238e67923344fce8c56e30a98100e92 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:37.291380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa063b0-dccf-4794-8f88-dc6a5e50abef None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:37.387933 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa063b0-dccf-4794-8f88-dc6a5e50abef None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:37.404540 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:37.421032 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d2071f-6b72-45c1-adf8-260d3f78fc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:37.421655 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2071f-6b72-45c1-adf8-260d3f78fc73 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:37.421938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2071f-6b72-45c1-adf8-260d3f78fc73 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:37.422212 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2071f-6b72-45c1-adf8-260d3f78fc73 None admin] PATH_INFO: `/v3/projects/522bea8c53b24b43b44ab47d12c6d122` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:37.423344 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d2071f-6b72-45c1-adf8-260d3f78fc73 None admin] RBAC: Authorizing `identity:get_project(project_id=522bea8c53b24b43b44ab47d12c6d122)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:37.424780 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d2071f-6b72-45c1-adf8-260d3f78fc73 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:37.426414 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 566/1122] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:37 2022] GET /identity/v3/projects/522bea8c53b24b43b44ab47d12c6d122 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:37.432366 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0470f2a2-5744-463a-9136-844ccb744c12 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:37.529543 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0470f2a2-5744-463a-9136-844ccb744c12 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:37.562653 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a54c00-ce8b-4617-b045-8342cc8dd5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:37.563302 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a54c00-ce8b-4617-b045-8342cc8dd5ce None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:37.563586 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a54c00-ce8b-4617-b045-8342cc8dd5ce None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:37.563862 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a54c00-ce8b-4617-b045-8342cc8dd5ce None admin] PATH_INFO: `/v3/projects/522bea8c53b24b43b44ab47d12c6d122/users/c238e67923344fce8c56e30a98100e92/roles/f2a6e936aa294323ba7087bf377276b2` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:37.565989 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a54c00-ce8b-4617-b045-8342cc8dd5ce None admin] RBAC: Authorizing `identity:create_grant(project_id=522bea8c53b24b43b44ab47d12c6d122, user_id=c238e67923344fce8c56e30a98100e92, role_id=f2a6e936aa294323ba7087bf377276b2)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:37.567437 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a54c00-ce8b-4617-b045-8342cc8dd5ce None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:37.580549 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 557/1123] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 1 16:22:37 2022] PUT /identity/v3/projects/522bea8c53b24b43b44ab47d12c6d122/users/c238e67923344fce8c56e30a98100e92/roles/f2a6e936aa294323ba7087bf377276b2 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:22:39.432025 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80ead9-64d0-49fb-bc72-2a97eae6377f None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:39.432591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80ead9-64d0-49fb-bc72-2a97eae6377f None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:39.432591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80ead9-64d0-49fb-bc72-2a97eae6377f None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:39.433920 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 567/1124] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:39.438930 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211907b-3d3e-44c3-9545-568ed4e75ff5 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:39.439319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211907b-3d3e-44c3-9545-568ed4e75ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:39.439485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211907b-3d3e-44c3-9545-568ed4e75ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:39.457345 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-9211907b-3d3e-44c3-9545-568ed4e75ff5 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:39.458628 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9211907b-3d3e-44c3-9545-568ed4e75ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:39.513712 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:39.526024 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 558/1125] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:22:39.794647 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e80ead9-64d0-49fb-bc72-2a97eae6377f None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:39.894885 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e80ead9-64d0-49fb-bc72-2a97eae6377f None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:39.927787 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed6f2e7-e158-4073-8a9f-d3ed50c8721c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:39.928415 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6f2e7-e158-4073-8a9f-d3ed50c8721c None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:39.928701 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6f2e7-e158-4073-8a9f-d3ed50c8721c None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:39.928980 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6f2e7-e158-4073-8a9f-d3ed50c8721c None admin] PATH_INFO: `/v3/roles/f2a6e936aa294323ba7087bf377276b2` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:39.930370 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed6f2e7-e158-4073-8a9f-d3ed50c8721c None admin] RBAC: Authorizing `identity:get_role(role_id=f2a6e936aa294323ba7087bf377276b2)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:39.931745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed6f2e7-e158-4073-8a9f-d3ed50c8721c None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:39.933049 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 568/1126] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:39 2022] GET /identity/v3/roles/f2a6e936aa294323ba7087bf377276b2 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:39.938929 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9211907b-3d3e-44c3-9545-568ed4e75ff5 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:40.035958 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9211907b-3d3e-44c3-9545-568ed4e75ff5 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:40.068613 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b04d5-3433-4515-b9c2-1b929a68d977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:40.069178 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b04d5-3433-4515-b9c2-1b929a68d977 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:40.069988 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b04d5-3433-4515-b9c2-1b929a68d977 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:40.070283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b04d5-3433-4515-b9c2-1b929a68d977 None admin] PATH_INFO: `/v3/users/c238e67923344fce8c56e30a98100e92` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:40.071477 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b04d5-3433-4515-b9c2-1b929a68d977 None admin] RBAC: Authorizing `identity:get_user(user_id=c238e67923344fce8c56e30a98100e92)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:40.072902 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b04d5-3433-4515-b9c2-1b929a68d977 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:40.074502 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 559/1127] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:39 2022] GET /identity/v3/users/c238e67923344fce8c56e30a98100e92 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:40.080118 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed6f2e7-e158-4073-8a9f-d3ed50c8721c None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:40.133915 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:40.177274 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed6f2e7-e158-4073-8a9f-d3ed50c8721c None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:40.209325 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa6b1b-f3ec-4811-b9e5-6343a477ed2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:40.210802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa6b1b-f3ec-4811-b9e5-6343a477ed2d None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:40.211235 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa6b1b-f3ec-4811-b9e5-6343a477ed2d None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:40.211515 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa6b1b-f3ec-4811-b9e5-6343a477ed2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:40.275776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa6b1b-f3ec-4811-b9e5-6343a477ed2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:40.277213 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa6b1b-f3ec-4811-b9e5-6343a477ed2d None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:40.327712 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:40.372362 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 569/1128] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:22:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:22:40.378188 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b04d5-3433-4515-b9c2-1b929a68d977 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:40.435253 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:40.476272 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b04d5-3433-4515-b9c2-1b929a68d977 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:40.508243 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f734a2f5-f3e7-4ff1-95a5-b3a4738496f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:40.508858 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734a2f5-f3e7-4ff1-95a5-b3a4738496f9 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:40.509139 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734a2f5-f3e7-4ff1-95a5-b3a4738496f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:40.509413 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734a2f5-f3e7-4ff1-95a5-b3a4738496f9 None admin] PATH_INFO: `/v3/projects/522bea8c53b24b43b44ab47d12c6d122` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:40.510532 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f734a2f5-f3e7-4ff1-95a5-b3a4738496f9 None admin] RBAC: Authorizing `identity:get_project(project_id=522bea8c53b24b43b44ab47d12c6d122)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:40.511932 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f734a2f5-f3e7-4ff1-95a5-b3a4738496f9 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:40.513523 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 560/1129] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:40 2022] GET /identity/v3/projects/522bea8c53b24b43b44ab47d12c6d122 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:40.519716 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa6b1b-f3ec-4811-b9e5-6343a477ed2d None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:40.614704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa6b1b-f3ec-4811-b9e5-6343a477ed2d None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:40.647188 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9846baf-6e42-4043-ad3a-0ffef79cedaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:40.648318 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9846baf-6e42-4043-ad3a-0ffef79cedaf None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:40.648599 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9846baf-6e42-4043-ad3a-0ffef79cedaf None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:40.648870 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9846baf-6e42-4043-ad3a-0ffef79cedaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:40.649584 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9846baf-6e42-4043-ad3a-0ffef79cedaf None admin] RBAC: Adding query filter params (role.id=f2a6e936aa294323ba7087bf377276b2, scope.project.id=522bea8c53b24b43b44ab47d12c6d122, user.id=c238e67923344fce8c56e30a98100e92) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:40.649894 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9846baf-6e42-4043-ad3a-0ffef79cedaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:40.651333 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9846baf-6e42-4043-ad3a-0ffef79cedaf None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:40.656835 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 570/1130] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:22:40 2022] GET /identity/v3/role_assignments?user.id=c238e67923344fce8c56e30a98100e92&scope.project.id=522bea8c53b24b43b44ab47d12c6d122&role.id=f2a6e936aa294323ba7087bf377276b2 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:42.525892 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209312a-7442-4f74-93b9-600048257932 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:42.526457 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209312a-7442-4f74-93b9-600048257932 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:42.526457 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209312a-7442-4f74-93b9-600048257932 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:42.527791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 561/1131] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:42.532820 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4b580-bf11-4fa3-a9b7-e0b6a7c9c06c None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:42.533201 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4b580-bf11-4fa3-a9b7-e0b6a7c9c06c None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:42.533386 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4b580-bf11-4fa3-a9b7-e0b6a7c9c06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:42.551319 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-39f4b580-bf11-4fa3-a9b7-e0b6a7c9c06c None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:42.552502 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-39f4b580-bf11-4fa3-a9b7-e0b6a7c9c06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:42.565700 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:42.617461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 571/1132] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:22:42.881670 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209312a-7442-4f74-93b9-600048257932 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:42.978412 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209312a-7442-4f74-93b9-600048257932 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:43.003298 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:43.012054 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a376f3bb-441a-4def-a24d-c1b28415dbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:43.012675 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376f3bb-441a-4def-a24d-c1b28415dbb1 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:43.012955 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376f3bb-441a-4def-a24d-c1b28415dbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:43.013227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376f3bb-441a-4def-a24d-c1b28415dbb1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:43.014337 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a376f3bb-441a-4def-a24d-c1b28415dbb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:43.015747 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a376f3bb-441a-4def-a24d-c1b28415dbb1 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:43.017278 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 562/1133] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:43.023014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4b580-bf11-4fa3-a9b7-e0b6a7c9c06c None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:43.118981 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4b580-bf11-4fa3-a9b7-e0b6a7c9c06c None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:43.151352 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a0c00d-409a-4db2-9b32-a5f07e1f9127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:43.151891 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0c00d-409a-4db2-9b32-a5f07e1f9127 None admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:43.152171 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0c00d-409a-4db2-9b32-a5f07e1f9127 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:43.152444 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0c00d-409a-4db2-9b32-a5f07e1f9127 None admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:43.153091 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a0c00d-409a-4db2-9b32-a5f07e1f9127 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:43.154611 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a0c00d-409a-4db2-9b32-a5f07e1f9127 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:43.174927 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:43.182512 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 572/1134] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Sep 1 16:22:43 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:22:45.075078 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a874a-ed09-45e2-8260-9fffd5280298 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:45.075639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a874a-ed09-45e2-8260-9fffd5280298 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:45.075639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a874a-ed09-45e2-8260-9fffd5280298 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:45.076959 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 563/1135] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:45.082032 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bbaa84-f0df-4215-a864-a18ac5b90d36 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:45.082453 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bbaa84-f0df-4215-a864-a18ac5b90d36 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:45.082634 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bbaa84-f0df-4215-a864-a18ac5b90d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:45.100963 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-93bbaa84-f0df-4215-a864-a18ac5b90d36 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:45.102143 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-93bbaa84-f0df-4215-a864-a18ac5b90d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:45.165853 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 573/1136] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:22:45.436771 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a874a-ed09-45e2-8260-9fffd5280298 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:45.534770 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a874a-ed09-45e2-8260-9fffd5280298 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:45.567238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6fdecf-ab98-4d20-868b-d2885681cab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:45.567849 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fdecf-ab98-4d20-868b-d2885681cab3 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:45.568130 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fdecf-ab98-4d20-868b-d2885681cab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:45.568406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fdecf-ab98-4d20-868b-d2885681cab3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:45.570059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6fdecf-ab98-4d20-868b-d2885681cab3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:45.571497 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6fdecf-ab98-4d20-868b-d2885681cab3 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:45.573020 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 564/1137] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:45.578584 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bbaa84-f0df-4215-a864-a18ac5b90d36 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:45.675019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bbaa84-f0df-4215-a864-a18ac5b90d36 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:45.703447 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:45.709463 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4be3a-afbd-41bd-b2c1-7f36afa757db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:45.710066 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4be3a-afbd-41bd-b2c1-7f36afa757db None admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:45.710387 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4be3a-afbd-41bd-b2c1-7f36afa757db None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:45.710668 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4be3a-afbd-41bd-b2c1-7f36afa757db None admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:45.711314 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf4be3a-afbd-41bd-b2c1-7f36afa757db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:45.712754 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf4be3a-afbd-41bd-b2c1-7f36afa757db None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:45.732732 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 574/1138] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 1 16:22:45 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:22:47.641728 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7c445-79fb-46ca-8c27-b233486cd24c None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:47.642260 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7c445-79fb-46ca-8c27-b233486cd24c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:47.642260 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7c445-79fb-46ca-8c27-b233486cd24c None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:47.643545 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 565/1139] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:47.648575 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf2001-a1d7-4095-a4cd-606f118a840c None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:47.648995 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf2001-a1d7-4095-a4cd-606f118a840c None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:47.649140 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf2001-a1d7-4095-a4cd-606f118a840c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:47.667438 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-72bf2001-a1d7-4095-a4cd-606f118a840c None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:47.668631 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-72bf2001-a1d7-4095-a4cd-606f118a840c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:47.734321 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 575/1140] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:22:47.998696 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7c445-79fb-46ca-8c27-b233486cd24c None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:48.063086 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:48.101365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7c445-79fb-46ca-8c27-b233486cd24c None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:48.133839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c483051-ddca-4cdb-b1cd-95933bc10efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:48.134474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c483051-ddca-4cdb-b1cd-95933bc10efd None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:48.134754 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c483051-ddca-4cdb-b1cd-95933bc10efd None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:48.135024 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c483051-ddca-4cdb-b1cd-95933bc10efd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:48.144783 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c483051-ddca-4cdb-b1cd-95933bc10efd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:48.150097 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-5c483051-ddca-4cdb-b1cd-95933bc10efd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:22:48.150097 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:48.150097 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:48.150097 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:48.150097 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:48.150097 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:48.150097 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:48.150097 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:48.150097 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:48.150097 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:48.150097 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:48.150097 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:48.150097 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:48.150097 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:48.150097 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:48.150097 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:48.151273 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:48.151273 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:48.151273 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:48.151273 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:48.151273 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:48.151273 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:48.151273 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:48.151273 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:48.151273 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:48.151273 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:48.151273 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:48.151273 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:48.151273 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:48.151273 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:48.151273 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:48.151273 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:48.152368 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:48.152368 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:48.152368 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:48.152368 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:48.152368 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:48.152368 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:48.152368 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:48.152368 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:48.152368 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:48.152368 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:48.152368 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:22:48.152368 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:48.152368 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 566/1141] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:22:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:48.157063 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf2001-a1d7-4095-a4cd-606f118a840c None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:48.252018 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf2001-a1d7-4095-a4cd-606f118a840c None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:48.268198 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:48.286513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3e8c7-1201-40dd-9fab-52aae89a3d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:48.287109 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3e8c7-1201-40dd-9fab-52aae89a3d3a None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:48.287418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3e8c7-1201-40dd-9fab-52aae89a3d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:48.287702 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3e8c7-1201-40dd-9fab-52aae89a3d3a None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:48.288358 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef3e8c7-1201-40dd-9fab-52aae89a3d3a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:48.288664 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef3e8c7-1201-40dd-9fab-52aae89a3d3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:48.290021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef3e8c7-1201-40dd-9fab-52aae89a3d3a None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:48.297821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 576/1142] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:22:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:48.303486 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c483051-ddca-4cdb-b1cd-95933bc10efd None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:48.402808 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c483051-ddca-4cdb-b1cd-95933bc10efd None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:48.424464 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:48.436454 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b9d70-c65e-4b3f-bd2a-651a8163e9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:48.437080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b9d70-c65e-4b3f-bd2a-651a8163e9b7 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:48.437361 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b9d70-c65e-4b3f-bd2a-651a8163e9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:48.437636 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b9d70-c65e-4b3f-bd2a-651a8163e9b7 None admin] PATH_INFO: `/v3/users/73e5524fa92344298bcf9735479a0693` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:48.451042 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9b9d70-c65e-4b3f-bd2a-651a8163e9b7 None admin] RBAC: Authorizing `identity:get_user(user_id=73e5524fa92344298bcf9735479a0693)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:48.452461 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9b9d70-c65e-4b3f-bd2a-651a8163e9b7 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:48.454030 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 567/1143] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:48 2022] GET /identity/v3/users/73e5524fa92344298bcf9735479a0693 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:48.459903 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3e8c7-1201-40dd-9fab-52aae89a3d3a None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:48.555394 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3e8c7-1201-40dd-9fab-52aae89a3d3a None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:48.587428 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec23d4-5161-4b4f-9d01-ee520df86ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:48.588019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec23d4-5161-4b4f-9d01-ee520df86ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:48.588303 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec23d4-5161-4b4f-9d01-ee520df86ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:48.588578 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec23d4-5161-4b4f-9d01-ee520df86ff3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:48.632379 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:48.653259 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec23d4-5161-4b4f-9d01-ee520df86ff3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:48.654739 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec23d4-5161-4b4f-9d01-ee520df86ff3 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:48.748664 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 577/1144] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:22:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:22:48.754445 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b9d70-c65e-4b3f-bd2a-651a8163e9b7 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:48.850405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b9d70-c65e-4b3f-bd2a-651a8163e9b7 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:48.884767 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def92158-1300-4ba6-877a-0374b9e02d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:48.885406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def92158-1300-4ba6-877a-0374b9e02d7e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:48.885686 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def92158-1300-4ba6-877a-0374b9e02d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:48.885956 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def92158-1300-4ba6-877a-0374b9e02d7e None admin] PATH_INFO: `/v3/projects/0ecc68c858034874bf46b9d61b6d1909` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:48.887093 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def92158-1300-4ba6-877a-0374b9e02d7e None admin] RBAC: Authorizing `identity:get_project(project_id=0ecc68c858034874bf46b9d61b6d1909)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:48.888537 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def92158-1300-4ba6-877a-0374b9e02d7e None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:48.890193 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 568/1145] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:48 2022] GET /identity/v3/projects/0ecc68c858034874bf46b9d61b6d1909 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:48.896505 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec23d4-5161-4b4f-9d01-ee520df86ff3 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:48.973879 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:48.995655 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec23d4-5161-4b4f-9d01-ee520df86ff3 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:49.030583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e76284-069d-480e-8f7a-f33fb017a4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:49.031190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e76284-069d-480e-8f7a-f33fb017a4b4 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:49.031472 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e76284-069d-480e-8f7a-f33fb017a4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:49.031746 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e76284-069d-480e-8f7a-f33fb017a4b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:49.032461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e76284-069d-480e-8f7a-f33fb017a4b4 None admin] RBAC: Adding query filter params (role.id=e8ba063905484d22993bcf04830091a3, scope.project.id=0ecc68c858034874bf46b9d61b6d1909, user.id=73e5524fa92344298bcf9735479a0693) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:49.032768 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e76284-069d-480e-8f7a-f33fb017a4b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:49.034194 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e76284-069d-480e-8f7a-f33fb017a4b4 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:49.039453 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 578/1146] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:22:48 2022] GET /identity/v3/role_assignments?user.id=73e5524fa92344298bcf9735479a0693&scope.project.id=0ecc68c858034874bf46b9d61b6d1909&role.id=e8ba063905484d22993bcf04830091a3 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:50.899607 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38175932-8078-427e-aa12-8516a702b200 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:50.900150 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38175932-8078-427e-aa12-8516a702b200 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:50.900150 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38175932-8078-427e-aa12-8516a702b200 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:50.901510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 569/1147] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:50.906662 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2655bc6-03a9-4e28-aa6a-733164b56c6c None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:50.907078 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2655bc6-03a9-4e28-aa6a-733164b56c6c None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:50.907224 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2655bc6-03a9-4e28-aa6a-733164b56c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:50.924887 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-d2655bc6-03a9-4e28-aa6a-733164b56c6c None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:50.926067 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-d2655bc6-03a9-4e28-aa6a-733164b56c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:50.989255 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 579/1148] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:22:51.255820 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38175932-8078-427e-aa12-8516a702b200 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:51.304867 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:51.354248 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38175932-8078-427e-aa12-8516a702b200 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:51.388040 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f0a35-50f6-4ad0-839a-d4a1d48af800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:51.388671 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f0a35-50f6-4ad0-839a-d4a1d48af800 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:51.388954 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f0a35-50f6-4ad0-839a-d4a1d48af800 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:51.389232 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f0a35-50f6-4ad0-839a-d4a1d48af800 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:51.399203 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f0a35-50f6-4ad0-839a-d4a1d48af800 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:51.404028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-5a2f0a35-50f6-4ad0-839a-d4a1d48af800 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:22:51.404028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:51.404028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:51.404028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:51.404028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:51.404028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:51.404028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:51.404028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:51.404028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:51.404028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:51.404028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:51.404028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:51.404028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:51.404028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:51.404028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:51.404028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:51.405096 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:51.405096 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:51.405096 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:51.405096 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:51.405096 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:51.405096 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:51.405096 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:51.405096 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:51.405096 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:51.405096 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:51.405096 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:51.405096 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:51.405096 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:51.405096 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:51.405096 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:51.405096 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:51.406331 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:51.406331 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:51.406331 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:51.406331 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:51.406331 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:51.406331 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:51.406331 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:51.406331 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:51.406331 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:51.406331 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:51.406331 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:22:51.406331 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:51.406331 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 570/1149] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:22:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:51.411077 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2655bc6-03a9-4e28-aa6a-733164b56c6c None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:51.479392 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:51.507812 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2655bc6-03a9-4e28-aa6a-733164b56c6c None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:51.540660 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ab8be-412a-4cbb-96a4-a9a2df66ea04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:51.541256 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ab8be-412a-4cbb-96a4-a9a2df66ea04 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:51.541537 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ab8be-412a-4cbb-96a4-a9a2df66ea04 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:51.541810 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ab8be-412a-4cbb-96a4-a9a2df66ea04 None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:51.542491 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ab8be-412a-4cbb-96a4-a9a2df66ea04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:51.542796 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ab8be-412a-4cbb-96a4-a9a2df66ea04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:51.544160 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ab8be-412a-4cbb-96a4-a9a2df66ea04 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:51.551846 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 580/1150] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:22:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:51.557451 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f0a35-50f6-4ad0-839a-d4a1d48af800 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:51.655017 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f0a35-50f6-4ad0-839a-d4a1d48af800 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:51.664507 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:51.689100 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb3ff49-478b-4830-899a-860fa4736469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:51.689718 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb3ff49-478b-4830-899a-860fa4736469 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:51.690003 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb3ff49-478b-4830-899a-860fa4736469 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:51.690309 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb3ff49-478b-4830-899a-860fa4736469 None admin] PATH_INFO: `/v3/users/73e5524fa92344298bcf9735479a0693` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:51.691443 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb3ff49-478b-4830-899a-860fa4736469 None admin] RBAC: Authorizing `identity:get_user(user_id=73e5524fa92344298bcf9735479a0693)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:51.692864 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb3ff49-478b-4830-899a-860fa4736469 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:51.694492 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 571/1151] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:51 2022] GET /identity/v3/users/73e5524fa92344298bcf9735479a0693 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:51.700341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ab8be-412a-4cbb-96a4-a9a2df66ea04 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:51.797191 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ab8be-412a-4cbb-96a4-a9a2df66ea04 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:51.829453 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd085d6f-0961-4809-839a-aca7424e83d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:51.830065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd085d6f-0961-4809-839a-aca7424e83d5 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:51.830375 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd085d6f-0961-4809-839a-aca7424e83d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:51.830654 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd085d6f-0961-4809-839a-aca7424e83d5 None admin] PATH_INFO: `/v3/projects/0ecc68c858034874bf46b9d61b6d1909` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:51.831750 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd085d6f-0961-4809-839a-aca7424e83d5 None admin] RBAC: Authorizing `identity:get_project(project_id=0ecc68c858034874bf46b9d61b6d1909)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:51.833169 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd085d6f-0961-4809-839a-aca7424e83d5 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:51.835342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 581/1152] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:51 2022] GET /identity/v3/projects/0ecc68c858034874bf46b9d61b6d1909 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:51.841051 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb3ff49-478b-4830-899a-860fa4736469 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:51.936890 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb3ff49-478b-4830-899a-860fa4736469 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:51.969995 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a8799-f6c7-4bf6-bc72-69f832f4a032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:51.970662 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a8799-f6c7-4bf6-bc72-69f832f4a032 None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:51.970946 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a8799-f6c7-4bf6-bc72-69f832f4a032 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:51.971223 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a8799-f6c7-4bf6-bc72-69f832f4a032 None admin] PATH_INFO: `/v3/projects/0ecc68c858034874bf46b9d61b6d1909/users/73e5524fa92344298bcf9735479a0693/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:51.973088 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a8799-f6c7-4bf6-bc72-69f832f4a032 None admin] RBAC: Authorizing `identity:create_grant(project_id=0ecc68c858034874bf46b9d61b6d1909, user_id=73e5524fa92344298bcf9735479a0693, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:51.974525 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a8799-f6c7-4bf6-bc72-69f832f4a032 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:51.987608 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 572/1153] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 1 16:22:51 2022] PUT /identity/v3/projects/0ecc68c858034874bf46b9d61b6d1909/users/73e5524fa92344298bcf9735479a0693/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:22:53.856964 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b0fd9-b971-40f9-a895-f362b33d3629 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:53.857524 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b0fd9-b971-40f9-a895-f362b33d3629 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:53.857524 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b0fd9-b971-40f9-a895-f362b33d3629 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:53.858901 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 582/1154] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:53.864752 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddecb36a-1933-4893-a090-f9502d30f6d0 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:53.865187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddecb36a-1933-4893-a090-f9502d30f6d0 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:53.865908 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddecb36a-1933-4893-a090-f9502d30f6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:53.885002 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-ddecb36a-1933-4893-a090-f9502d30f6d0 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:53.886183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ddecb36a-1933-4893-a090-f9502d30f6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:53.913535 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:53.952572 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 573/1155] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:22:54.219056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b0fd9-b971-40f9-a895-f362b33d3629 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:54.221900 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:54.319301 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b0fd9-b971-40f9-a895-f362b33d3629 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:54.351708 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ea7b0-e64c-417d-b12e-1be11ac464ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:54.352338 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ea7b0-e64c-417d-b12e-1be11ac464ed None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:54.352617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ea7b0-e64c-417d-b12e-1be11ac464ed None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:54.352889 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ea7b0-e64c-417d-b12e-1be11ac464ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:54.362919 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ea7b0-e64c-417d-b12e-1be11ac464ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:54.368462 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-214ea7b0-e64c-417d-b12e-1be11ac464ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:22:54.368462 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:54.368462 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:54.368462 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:54.368462 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:54.368462 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:54.368462 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:54.368462 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:54.368462 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:54.368462 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:54.368462 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:54.368462 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:54.368462 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:54.368462 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:54.368462 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:54.368462 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:54.369536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:54.369536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:54.369536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:54.369536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:54.369536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:54.369536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:54.369536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:54.369536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:54.369536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:54.369536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:54.369536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:54.369536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:54.369536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:54.369536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:54.369536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:54.369536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:54.370580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:54.370580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:54.370580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:54.370580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:54.370580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:54.370580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:54.370580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:54.370580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:54.370580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:54.370580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:54.370580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:22:54.370580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:22:54.370580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 583/1156] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:22:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:54.375524 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddecb36a-1933-4893-a090-f9502d30f6d0 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:54.471328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddecb36a-1933-4893-a090-f9502d30f6d0 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:54.504383 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ad66e-ba9f-4ab5-97ae-1a0b75f9a482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:54.505010 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ad66e-ba9f-4ab5-97ae-1a0b75f9a482 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:54.505312 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ad66e-ba9f-4ab5-97ae-1a0b75f9a482 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:54.505586 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ad66e-ba9f-4ab5-97ae-1a0b75f9a482 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:54.506268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ad66e-ba9f-4ab5-97ae-1a0b75f9a482 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:54.506570 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ad66e-ba9f-4ab5-97ae-1a0b75f9a482 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:54.507915 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ad66e-ba9f-4ab5-97ae-1a0b75f9a482 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:54.515723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 574/1157] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:22:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:54.521239 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ea7b0-e64c-417d-b12e-1be11ac464ed None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:54.580993 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:54.619038 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ea7b0-e64c-417d-b12e-1be11ac464ed None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:54.651812 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef5e13-6b3c-46cc-b85c-ee7917bfb759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:54.652427 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef5e13-6b3c-46cc-b85c-ee7917bfb759 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:54.652715 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef5e13-6b3c-46cc-b85c-ee7917bfb759 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:54.652993 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef5e13-6b3c-46cc-b85c-ee7917bfb759 None admin] PATH_INFO: `/v3/users/73e5524fa92344298bcf9735479a0693` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:54.654134 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ef5e13-6b3c-46cc-b85c-ee7917bfb759 None admin] RBAC: Authorizing `identity:get_user(user_id=73e5524fa92344298bcf9735479a0693)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:54.655603 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ef5e13-6b3c-46cc-b85c-ee7917bfb759 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:54.657159 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 584/1158] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:54 2022] GET /identity/v3/users/73e5524fa92344298bcf9735479a0693 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:54.662912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ad66e-ba9f-4ab5-97ae-1a0b75f9a482 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:54.690866 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:54.760355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ad66e-ba9f-4ab5-97ae-1a0b75f9a482 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:54.792152 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090bfa3d-8a49-4aaa-a77d-3c49a04f6181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:54.792744 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bfa3d-8a49-4aaa-a77d-3c49a04f6181 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:54.793023 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bfa3d-8a49-4aaa-a77d-3c49a04f6181 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:54.793293 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bfa3d-8a49-4aaa-a77d-3c49a04f6181 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:54.857480 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090bfa3d-8a49-4aaa-a77d-3c49a04f6181 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:54.858934 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090bfa3d-8a49-4aaa-a77d-3c49a04f6181 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:54.885551 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:54.953618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 575/1159] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:22:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:22:54.959450 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef5e13-6b3c-46cc-b85c-ee7917bfb759 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:55.056145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef5e13-6b3c-46cc-b85c-ee7917bfb759 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:55.082987 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:55.088983 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db1891-7a44-4696-8b49-5dc6348c6968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:55.089597 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db1891-7a44-4696-8b49-5dc6348c6968 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:55.089878 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db1891-7a44-4696-8b49-5dc6348c6968 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:55.090149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db1891-7a44-4696-8b49-5dc6348c6968 None admin] PATH_INFO: `/v3/projects/0ecc68c858034874bf46b9d61b6d1909` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:55.091288 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db1891-7a44-4696-8b49-5dc6348c6968 None admin] RBAC: Authorizing `identity:get_project(project_id=0ecc68c858034874bf46b9d61b6d1909)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:55.092702 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db1891-7a44-4696-8b49-5dc6348c6968 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:55.094325 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 585/1160] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:54 2022] GET /identity/v3/projects/0ecc68c858034874bf46b9d61b6d1909 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:55.100233 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090bfa3d-8a49-4aaa-a77d-3c49a04f6181 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:55.196067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090bfa3d-8a49-4aaa-a77d-3c49a04f6181 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:55.221653 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:55.230277 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df191b7-4907-4b59-a38c-592ed85f6aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:55.230878 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df191b7-4907-4b59-a38c-592ed85f6aca None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:55.231163 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df191b7-4907-4b59-a38c-592ed85f6aca None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:55.231440 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df191b7-4907-4b59-a38c-592ed85f6aca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:55.232164 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df191b7-4907-4b59-a38c-592ed85f6aca None admin] RBAC: Adding query filter params (role.id=e8ba063905484d22993bcf04830091a3, scope.project.id=0ecc68c858034874bf46b9d61b6d1909, user.id=73e5524fa92344298bcf9735479a0693) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:55.232480 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df191b7-4907-4b59-a38c-592ed85f6aca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:55.233895 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df191b7-4907-4b59-a38c-592ed85f6aca None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:55.239547 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 576/1161] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:22:55 2022] GET /identity/v3/role_assignments?user.id=73e5524fa92344298bcf9735479a0693&scope.project.id=0ecc68c858034874bf46b9d61b6d1909&role.id=e8ba063905484d22993bcf04830091a3 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:57.141715 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb76852-896f-428d-b75d-75e8505a5b8b None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:57.142266 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb76852-896f-428d-b75d-75e8505a5b8b None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:57.142266 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb76852-896f-428d-b75d-75e8505a5b8b None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:57.143603 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 586/1162] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:57.149422 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfa192-d951-4882-a1f8-a7392f58e28a None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:57.149846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfa192-d951-4882-a1f8-a7392f58e28a None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:57.149980 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfa192-d951-4882-a1f8-a7392f58e28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:57.167946 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-ecdfa192-d951-4882-a1f8-a7392f58e28a None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:57.169142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ecdfa192-d951-4882-a1f8-a7392f58e28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:57.234392 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 577/1163] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:22:57.499765 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb76852-896f-428d-b75d-75e8505a5b8b None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:57.598739 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb76852-896f-428d-b75d-75e8505a5b8b None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:57.631931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517a1525-2562-4776-9745-55a0198c7878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:57.633131 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a1525-2562-4776-9745-55a0198c7878 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:57.633413 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a1525-2562-4776-9745-55a0198c7878 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:57.633687 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a1525-2562-4776-9745-55a0198c7878 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:57.696331 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:57.699723 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517a1525-2562-4776-9745-55a0198c7878 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:57.701659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517a1525-2562-4776-9745-55a0198c7878 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:57.796591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 587/1164] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:22:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:22:57.802649 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdfa192-d951-4882-a1f8-a7392f58e28a None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:57.897500 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdfa192-d951-4882-a1f8-a7392f58e28a None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:57.919505 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:57.931699 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1a289-3828-4ab3-bd8e-deb29ef2b198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:57.932318 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1a289-3828-4ab3-bd8e-deb29ef2b198 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:57.932600 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1a289-3828-4ab3-bd8e-deb29ef2b198 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:57.932891 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1a289-3828-4ab3-bd8e-deb29ef2b198 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:57.939409 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b1a289-3828-4ab3-bd8e-deb29ef2b198 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:57.940835 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b1a289-3828-4ab3-bd8e-deb29ef2b198 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:57.952093 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-89b1a289-3828-4ab3-bd8e-deb29ef2b198 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 16:22:57.952093 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:57.952093 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 01 16:22:57.952093 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 01 16:22:57.952093 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:22:57.952093 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:22:57.952093 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:22:57.952093 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:22:57.952093 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 01 16:22:57.952093 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:57.952093 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:22:57.952093 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:57.952093 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:57.952093 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:57.952093 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:57.952093 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:57.952093 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:57.953280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:57.953280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:57.953280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:57.953280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:57.953280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:57.953280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:57.953280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:22:57.953280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:22:57.953280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:22:57.953280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:22:57.953280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:57.953280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:57.953280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:57.953280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:57.953280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:57.953280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:57.954319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:57.954319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:57.954319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:57.954319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:57.954319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:57.954319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:57.954319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:57.954319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:57.954319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:57.954319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:57.954319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 01 16:22:57.954319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:22:57.954319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 16:22:57.954319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:22:57.957911 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 578/1165] 10.222.0.18 () {56 vars in 1117 bytes} [Thu Sep 1 16:22:57 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:57.959062 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517a1525-2562-4776-9745-55a0198c7878 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:58.050931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:58.055573 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517a1525-2562-4776-9745-55a0198c7878 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:58.087727 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09794254-ad18-4d5f-a175-0503c8a66a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:58.088340 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09794254-ad18-4d5f-a175-0503c8a66a4b None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:58.088618 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09794254-ad18-4d5f-a175-0503c8a66a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:58.088888 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09794254-ad18-4d5f-a175-0503c8a66a4b None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:58.089513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09794254-ad18-4d5f-a175-0503c8a66a4b None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:58.089814 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09794254-ad18-4d5f-a175-0503c8a66a4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:58.091187 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09794254-ad18-4d5f-a175-0503c8a66a4b None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:58.097961 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 588/1166] 10.222.0.18 () {56 vars in 1115 bytes} [Thu Sep 1 16:22:57 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:59.953467 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2214c-9369-4a09-add3-46d51c5cd3fb None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:59.954026 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2214c-9369-4a09-add3-46d51c5cd3fb None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:59.954026 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2214c-9369-4a09-add3-46d51c5cd3fb None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:59.955396 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 579/1167] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:59.960874 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa1742-99ba-4a43-b34d-2f2cc1c08340 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:59.961292 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa1742-99ba-4a43-b34d-2f2cc1c08340 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:59.961427 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa1742-99ba-4a43-b34d-2f2cc1c08340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:59.979894 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-62fa1742-99ba-4a43-b34d-2f2cc1c08340 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:59.981092 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-62fa1742-99ba-4a43-b34d-2f2cc1c08340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:00.045571 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 589/1168] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:23:00.311087 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2214c-9369-4a09-add3-46d51c5cd3fb None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:00.411153 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2214c-9369-4a09-add3-46d51c5cd3fb None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:00.443693 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275f97d-336f-4860-8115-ba43ccdc8769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:00.444291 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275f97d-336f-4860-8115-ba43ccdc8769 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:00.445161 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275f97d-336f-4860-8115-ba43ccdc8769 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:00.445440 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275f97d-336f-4860-8115-ba43ccdc8769 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:00.446101 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275f97d-336f-4860-8115-ba43ccdc8769 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:00.447498 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275f97d-336f-4860-8115-ba43ccdc8769 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:00.462983 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 580/1169] 10.222.0.18 () {60 vars in 1138 bytes} [Thu Sep 1 16:23:00 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:23:02.363632 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2e78b-197e-4539-84aa-d66d8f860c99 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:02.364212 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2e78b-197e-4539-84aa-d66d8f860c99 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:02.364212 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2e78b-197e-4539-84aa-d66d8f860c99 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:02.365603 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 590/1170] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:23:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:02.371265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473eaa37-15c2-4750-83af-0cf8082d3464 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:02.371690 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473eaa37-15c2-4750-83af-0cf8082d3464 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:02.371831 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473eaa37-15c2-4750-83af-0cf8082d3464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:02.390078 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-473eaa37-15c2-4750-83af-0cf8082d3464 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:02.391308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-473eaa37-15c2-4750-83af-0cf8082d3464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:02.431279 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:02.462958 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 581/1171] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:23:02.727183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2e78b-197e-4539-84aa-d66d8f860c99 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:02.816698 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:02.826969 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2e78b-197e-4539-84aa-d66d8f860c99 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:02.859510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8b803-8d4d-40ec-8d99-908ec2b8451a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:02.860138 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8b803-8d4d-40ec-8d99-908ec2b8451a None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:02.860425 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8b803-8d4d-40ec-8d99-908ec2b8451a None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:02.860705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8b803-8d4d-40ec-8d99-908ec2b8451a None admin] PATH_INFO: `/v3/domains/635c83fd68ed4541931971996750c32e` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:02.871591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d8b803-8d4d-40ec-8d99-908ec2b8451a None admin] RBAC: Authorizing `identity:get_domain(domain_id=635c83fd68ed4541931971996750c32e)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:02.873039 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d8b803-8d4d-40ec-8d99-908ec2b8451a None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:02.874640 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 591/1172] 10.222.0.18 () {56 vars in 1183 bytes} [Thu Sep 1 16:23:02 2022] GET /identity/v3/domains/635c83fd68ed4541931971996750c32e => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:02.880357 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473eaa37-15c2-4750-83af-0cf8082d3464 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:02.975537 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473eaa37-15c2-4750-83af-0cf8082d3464 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:03.008906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe4593d-384f-42ba-8187-446c7852df0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:03.009446 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4593d-384f-42ba-8187-446c7852df0e None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:03.009722 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4593d-384f-42ba-8187-446c7852df0e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:03.009991 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4593d-384f-42ba-8187-446c7852df0e None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:03.010649 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe4593d-384f-42ba-8187-446c7852df0e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:03.012070 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe4593d-384f-42ba-8187-446c7852df0e None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:03.037126 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 582/1173] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Sep 1 16:23:02 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:23:04.922843 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478396e-5f88-4437-b4df-09eae7b3c9dd None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:04.923391 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478396e-5f88-4437-b4df-09eae7b3c9dd None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:04.923391 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478396e-5f88-4437-b4df-09eae7b3c9dd None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:04.924742 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 592/1174] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:04.929698 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0066f-e1b6-4669-9bc2-00cf17a428e6 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:04.930123 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0066f-e1b6-4669-9bc2-00cf17a428e6 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:04.930268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0066f-e1b6-4669-9bc2-00cf17a428e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:04.948048 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-51b0066f-e1b6-4669-9bc2-00cf17a428e6 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:04.949231 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-51b0066f-e1b6-4669-9bc2-00cf17a428e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:04.966404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:05.015541 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 583/1175] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:23:05.282854 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6478396e-5f88-4437-b4df-09eae7b3c9dd None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:05.380989 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6478396e-5f88-4437-b4df-09eae7b3c9dd None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:05.414184 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66926e6-37e2-4b94-9489-fc88bd5c95f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:05.414841 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66926e6-37e2-4b94-9489-fc88bd5c95f3 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:05.415125 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66926e6-37e2-4b94-9489-fc88bd5c95f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:05.415402 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66926e6-37e2-4b94-9489-fc88bd5c95f3 None admin] PATH_INFO: `/v3/domains/635c83fd68ed4541931971996750c32e` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:05.416502 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66926e6-37e2-4b94-9489-fc88bd5c95f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=635c83fd68ed4541931971996750c32e)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:05.417931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66926e6-37e2-4b94-9489-fc88bd5c95f3 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:05.419512 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 593/1176] 10.222.0.18 () {56 vars in 1183 bytes} [Thu Sep 1 16:23:05 2022] GET /identity/v3/domains/635c83fd68ed4541931971996750c32e => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:05.425150 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0066f-e1b6-4669-9bc2-00cf17a428e6 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:05.520948 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0066f-e1b6-4669-9bc2-00cf17a428e6 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:05.553226 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69036953-6ed8-4108-b4c1-170fc3677615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:05.553809 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69036953-6ed8-4108-b4c1-170fc3677615 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:05.554111 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69036953-6ed8-4108-b4c1-170fc3677615 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:05.554420 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69036953-6ed8-4108-b4c1-170fc3677615 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:05.555061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69036953-6ed8-4108-b4c1-170fc3677615 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:05.556483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69036953-6ed8-4108-b4c1-170fc3677615 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:05.577483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 584/1177] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 1 16:23:05 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:23:07.505809 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55dc332-a92f-41bc-bd91-7b0a55df918c None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:07.506363 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55dc332-a92f-41bc-bd91-7b0a55df918c None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:07.506458 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55dc332-a92f-41bc-bd91-7b0a55df918c None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:07.508227 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 594/1178] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:07.513336 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a779a57a-91ba-4fc0-99a3-12d47b39ed61 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:07.513729 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a779a57a-91ba-4fc0-99a3-12d47b39ed61 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:07.513885 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a779a57a-91ba-4fc0-99a3-12d47b39ed61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:07.532167 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-a779a57a-91ba-4fc0-99a3-12d47b39ed61 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:07.533343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a779a57a-91ba-4fc0-99a3-12d47b39ed61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:07.536056 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:07.598639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 585/1179] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:23:07.864923 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55dc332-a92f-41bc-bd91-7b0a55df918c None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:07.887215 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:07.964591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55dc332-a92f-41bc-bd91-7b0a55df918c None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:07.997467 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206e8a2-c4d8-439d-bdfb-0cc8793dc8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:07.998085 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206e8a2-c4d8-439d-bdfb-0cc8793dc8bc None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:07.998398 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206e8a2-c4d8-439d-bdfb-0cc8793dc8bc None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:07.998677 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206e8a2-c4d8-439d-bdfb-0cc8793dc8bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:08.009133 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206e8a2-c4d8-439d-bdfb-0cc8793dc8bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:08.013916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-b206e8a2-c4d8-439d-bdfb-0cc8793dc8bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:23:08.013916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:08.013916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:08.013916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:08.013916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:08.013916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:08.013916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:08.013916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:08.013916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:08.013916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:08.013916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:08.013916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:08.013916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:08.013916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:08.013916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:08.013916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:08.015092 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:08.015092 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:08.015092 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:08.015092 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:08.015092 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:08.015092 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:08.015092 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:08.015092 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:08.015092 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:08.015092 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:08.015092 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:08.015092 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:08.015092 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:08.015092 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:08.015092 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:08.015092 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:08.016168 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:08.016168 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:08.016168 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:08.016168 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:08.016168 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:08.016168 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:08.016168 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:08.016168 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:08.016168 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:08.016168 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:08.016168 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:23:08.016168 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:23:08.016168 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 595/1180] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:23:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:08.020921 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a779a57a-91ba-4fc0-99a3-12d47b39ed61 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:08.115802 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a779a57a-91ba-4fc0-99a3-12d47b39ed61 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:08.148809 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eeb612-7a58-414d-99b2-ff42313813e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:08.149404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eeb612-7a58-414d-99b2-ff42313813e0 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:08.149680 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eeb612-7a58-414d-99b2-ff42313813e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:08.149949 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eeb612-7a58-414d-99b2-ff42313813e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:08.150629 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eeb612-7a58-414d-99b2-ff42313813e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:08.150933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eeb612-7a58-414d-99b2-ff42313813e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:08.152276 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eeb612-7a58-414d-99b2-ff42313813e0 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:08.159975 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 586/1181] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:23:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:08.165455 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206e8a2-c4d8-439d-bdfb-0cc8793dc8bc None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:08.244783 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:08.264495 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206e8a2-c4d8-439d-bdfb-0cc8793dc8bc None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:08.297294 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd204aed-19d5-4943-a184-39f786fbd7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:08.297907 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd204aed-19d5-4943-a184-39f786fbd7c2 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:08.298191 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd204aed-19d5-4943-a184-39f786fbd7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:08.298499 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd204aed-19d5-4943-a184-39f786fbd7c2 None admin] PATH_INFO: `/v3/users/af76035096944c6089b84898f4dfd914` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:08.311547 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd204aed-19d5-4943-a184-39f786fbd7c2 None admin] RBAC: Authorizing `identity:get_user(user_id=af76035096944c6089b84898f4dfd914)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:08.312992 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd204aed-19d5-4943-a184-39f786fbd7c2 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:08.314589 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 596/1182] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:23:08 2022] GET /identity/v3/users/af76035096944c6089b84898f4dfd914 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:08.320300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eeb612-7a58-414d-99b2-ff42313813e0 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:08.323104 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:08.417128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eeb612-7a58-414d-99b2-ff42313813e0 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:08.449007 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea41d8-3f15-4d75-8cf7-6998c1756830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:08.449602 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea41d8-3f15-4d75-8cf7-6998c1756830 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:08.449878 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea41d8-3f15-4d75-8cf7-6998c1756830 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:08.450147 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea41d8-3f15-4d75-8cf7-6998c1756830 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:08.513976 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ea41d8-3f15-4d75-8cf7-6998c1756830 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:08.515443 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ea41d8-3f15-4d75-8cf7-6998c1756830 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:08.517162 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:08.608933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 587/1183] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:23:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 16:23:08.614838 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd204aed-19d5-4943-a184-39f786fbd7c2 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:08.710664 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd204aed-19d5-4943-a184-39f786fbd7c2 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:08.743163 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dbddaf-1c51-48dd-b9a3-f7e70241e841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:08.743778 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dbddaf-1c51-48dd-b9a3-f7e70241e841 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:08.744057 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dbddaf-1c51-48dd-b9a3-f7e70241e841 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:08.744330 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dbddaf-1c51-48dd-b9a3-f7e70241e841 None admin] PATH_INFO: `/v3/projects/06db84a05016481382ec1d9400c953b9` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:08.745418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dbddaf-1c51-48dd-b9a3-f7e70241e841 None admin] RBAC: Authorizing `identity:get_project(project_id=06db84a05016481382ec1d9400c953b9)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:08.746872 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dbddaf-1c51-48dd-b9a3-f7e70241e841 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:08.748455 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 597/1184] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:23:08 2022] GET /identity/v3/projects/06db84a05016481382ec1d9400c953b9 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:08.754369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea41d8-3f15-4d75-8cf7-6998c1756830 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:08.849133 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea41d8-3f15-4d75-8cf7-6998c1756830 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:08.851267 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:08.883617 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa3f3b-37d4-4b00-9c50-824a7801492a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:08.884218 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa3f3b-37d4-4b00-9c50-824a7801492a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:08.884527 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa3f3b-37d4-4b00-9c50-824a7801492a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:08.884832 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa3f3b-37d4-4b00-9c50-824a7801492a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:08.885556 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa3f3b-37d4-4b00-9c50-824a7801492a None admin] RBAC: Adding query filter params (role.id=e8ba063905484d22993bcf04830091a3, scope.project.id=06db84a05016481382ec1d9400c953b9, user.id=af76035096944c6089b84898f4dfd914) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:08.885867 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa3f3b-37d4-4b00-9c50-824a7801492a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:08.887306 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa3f3b-37d4-4b00-9c50-824a7801492a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:08.892535 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 588/1185] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:23:08 2022] GET /identity/v3/role_assignments?user.id=af76035096944c6089b84898f4dfd914&scope.project.id=06db84a05016481382ec1d9400c953b9&role.id=e8ba063905484d22993bcf04830091a3 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:10.731832 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec3a0c-10c5-4047-898f-cf0b4e377159 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:10.732366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec3a0c-10c5-4047-898f-cf0b4e377159 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:10.732366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec3a0c-10c5-4047-898f-cf0b4e377159 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:10.733623 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 598/1186] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:10.738803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772bd0a4-eaa3-4ef4-a970-d7b6ed6f9233 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:10.739214 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772bd0a4-eaa3-4ef4-a970-d7b6ed6f9233 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:10.739352 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772bd0a4-eaa3-4ef4-a970-d7b6ed6f9233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:10.757697 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-772bd0a4-eaa3-4ef4-a970-d7b6ed6f9233 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:10.758902 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-772bd0a4-eaa3-4ef4-a970-d7b6ed6f9233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:10.823011 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 589/1187] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:23:11.084994 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec3a0c-10c5-4047-898f-cf0b4e377159 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:11.094102 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:11.184058 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec3a0c-10c5-4047-898f-cf0b4e377159 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:11.216174 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba83d1e-c04d-4061-8f27-ede3daec1cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:11.216782 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba83d1e-c04d-4061-8f27-ede3daec1cff None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:11.217067 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba83d1e-c04d-4061-8f27-ede3daec1cff None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:11.217337 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba83d1e-c04d-4061-8f27-ede3daec1cff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:11.227009 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba83d1e-c04d-4061-8f27-ede3daec1cff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:11.231699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-2ba83d1e-c04d-4061-8f27-ede3daec1cff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:23:11.231699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:11.231699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:11.231699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:11.231699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:11.231699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:11.231699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:11.231699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:11.231699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:11.231699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:11.231699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:11.231699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:11.231699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:11.231699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:11.231699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:11.231699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:11.232822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:11.232822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:11.232822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:11.232822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:11.232822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:11.232822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:11.232822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:11.232822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:11.232822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:11.232822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:11.232822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:11.232822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:11.232822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:11.232822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:11.232822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:11.232822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:11.233718 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:11.233718 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:11.233718 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:11.233718 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:11.233718 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:11.233718 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:11.233718 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:11.233718 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:11.233718 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:11.233718 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:11.233718 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:23:11.233718 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:23:11.233718 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 599/1188] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:23:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:11.238654 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772bd0a4-eaa3-4ef4-a970-d7b6ed6f9233 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:11.329352 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:11.335528 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772bd0a4-eaa3-4ef4-a970-d7b6ed6f9233 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:11.369609 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6da7fc-6121-4a5d-960d-c8efd96d073b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:11.370331 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6da7fc-6121-4a5d-960d-c8efd96d073b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:11.370684 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6da7fc-6121-4a5d-960d-c8efd96d073b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:11.370963 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6da7fc-6121-4a5d-960d-c8efd96d073b None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:11.371626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6da7fc-6121-4a5d-960d-c8efd96d073b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:11.371931 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6da7fc-6121-4a5d-960d-c8efd96d073b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:11.373295 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6da7fc-6121-4a5d-960d-c8efd96d073b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:11.381287 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 590/1189] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:23:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:11.387064 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba83d1e-c04d-4061-8f27-ede3daec1cff None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:11.450545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:11.483977 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba83d1e-c04d-4061-8f27-ede3daec1cff None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:11.517480 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1e2ac-b542-4aec-8e86-fabda4f44a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:11.518089 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1e2ac-b542-4aec-8e86-fabda4f44a40 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:11.518403 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1e2ac-b542-4aec-8e86-fabda4f44a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:11.518677 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1e2ac-b542-4aec-8e86-fabda4f44a40 None admin] PATH_INFO: `/v3/users/af76035096944c6089b84898f4dfd914` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:11.519759 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e1e2ac-b542-4aec-8e86-fabda4f44a40 None admin] RBAC: Authorizing `identity:get_user(user_id=af76035096944c6089b84898f4dfd914)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:11.521177 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e1e2ac-b542-4aec-8e86-fabda4f44a40 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:11.522769 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 600/1190] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:23:11 2022] GET /identity/v3/users/af76035096944c6089b84898f4dfd914 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:11.528379 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6da7fc-6121-4a5d-960d-c8efd96d073b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:11.625861 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6da7fc-6121-4a5d-960d-c8efd96d073b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:11.657986 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b0e83a-00ed-462c-b5a7-4a955c4f5f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:11.658632 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0e83a-00ed-462c-b5a7-4a955c4f5f5a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:11.658916 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0e83a-00ed-462c-b5a7-4a955c4f5f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:11.659193 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0e83a-00ed-462c-b5a7-4a955c4f5f5a None admin] PATH_INFO: `/v3/projects/06db84a05016481382ec1d9400c953b9` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:11.660307 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b0e83a-00ed-462c-b5a7-4a955c4f5f5a None admin] RBAC: Authorizing `identity:get_project(project_id=06db84a05016481382ec1d9400c953b9)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:11.661719 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b0e83a-00ed-462c-b5a7-4a955c4f5f5a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:11.663927 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 591/1191] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:23:11 2022] GET /identity/v3/projects/06db84a05016481382ec1d9400c953b9 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:11.669598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1e2ac-b542-4aec-8e86-fabda4f44a40 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:11.776781 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1e2ac-b542-4aec-8e86-fabda4f44a40 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:11.808793 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec68195-080f-425c-a69f-e8faef47ce67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:11.809439 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec68195-080f-425c-a69f-e8faef47ce67 None admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:11.809721 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec68195-080f-425c-a69f-e8faef47ce67 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:11.809996 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec68195-080f-425c-a69f-e8faef47ce67 None admin] PATH_INFO: `/v3/projects/06db84a05016481382ec1d9400c953b9/users/af76035096944c6089b84898f4dfd914/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:11.811891 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec68195-080f-425c-a69f-e8faef47ce67 None admin] RBAC: Authorizing `identity:create_grant(project_id=06db84a05016481382ec1d9400c953b9, user_id=af76035096944c6089b84898f4dfd914, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:11.813327 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec68195-080f-425c-a69f-e8faef47ce67 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:11.815536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:11.825957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 601/1192] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 1 16:23:11 2022] PUT /identity/v3/projects/06db84a05016481382ec1d9400c953b9/users/af76035096944c6089b84898f4dfd914/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:23:13.674649 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461db7b1-9205-4bd3-9625-da054f066d02 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:13.675162 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461db7b1-9205-4bd3-9625-da054f066d02 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:13.675162 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461db7b1-9205-4bd3-9625-da054f066d02 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:13.676449 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 592/1193] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:23:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:13.682156 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebefad-cad0-4d01-ae0a-290661c7e2fb None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:13.682583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebefad-cad0-4d01-ae0a-290661c7e2fb None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:13.682740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebefad-cad0-4d01-ae0a-290661c7e2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:13.700695 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-eeebefad-cad0-4d01-ae0a-290661c7e2fb None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:13.701902 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-eeebefad-cad0-4d01-ae0a-290661c7e2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:13.767258 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 602/1194] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:23:14.035995 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461db7b1-9205-4bd3-9625-da054f066d02 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:14.063642 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:14.135411 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461db7b1-9205-4bd3-9625-da054f066d02 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:14.168075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e77a799-24fb-42d2-8fd5-f1a16b89fe79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:14.168693 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e77a799-24fb-42d2-8fd5-f1a16b89fe79 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:14.168976 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e77a799-24fb-42d2-8fd5-f1a16b89fe79 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:14.169252 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e77a799-24fb-42d2-8fd5-f1a16b89fe79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:14.179486 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e77a799-24fb-42d2-8fd5-f1a16b89fe79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:14.184190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-1e77a799-24fb-42d2-8fd5-f1a16b89fe79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:23:14.184190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:14.184190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:14.184190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:14.184190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:14.184190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:14.184190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:14.184190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:14.184190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:14.184190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:14.184190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:14.184190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:14.184190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:14.184190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:14.184190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:14.184190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:14.185250 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:14.185250 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:14.185250 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:14.185250 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:14.185250 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:14.185250 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:14.185250 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:14.185250 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:14.185250 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:14.185250 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:14.185250 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:14.185250 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:14.185250 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:14.185250 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:14.185250 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:14.185250 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:14.186090 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:14.186090 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:14.186090 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:14.186090 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:14.186090 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:14.186090 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:14.186090 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:14.186090 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:14.186090 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:14.186090 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:14.186090 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:23:14.186090 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:23:14.186090 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 593/1195] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:23:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:14.191266 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeebefad-cad0-4d01-ae0a-290661c7e2fb None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:14.287529 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeebefad-cad0-4d01-ae0a-290661c7e2fb None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:14.309678 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:14.321220 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b9532-0a7f-4587-8d2c-0a285b8f6c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:14.321820 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b9532-0a7f-4587-8d2c-0a285b8f6c5e None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:14.322104 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b9532-0a7f-4587-8d2c-0a285b8f6c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:14.322412 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b9532-0a7f-4587-8d2c-0a285b8f6c5e None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:14.323070 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b9532-0a7f-4587-8d2c-0a285b8f6c5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:14.323378 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b9532-0a7f-4587-8d2c-0a285b8f6c5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:14.324733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b9532-0a7f-4587-8d2c-0a285b8f6c5e None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:14.332586 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 603/1196] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:23:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:14.338803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e77a799-24fb-42d2-8fd5-f1a16b89fe79 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:14.421588 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:14.436288 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e77a799-24fb-42d2-8fd5-f1a16b89fe79 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:14.469364 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bffeed5-c3f7-430b-af61-19355240bc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:14.469983 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bffeed5-c3f7-430b-af61-19355240bc2e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:14.470286 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bffeed5-c3f7-430b-af61-19355240bc2e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:14.470568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bffeed5-c3f7-430b-af61-19355240bc2e None admin] PATH_INFO: `/v3/users/af76035096944c6089b84898f4dfd914` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:14.471673 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bffeed5-c3f7-430b-af61-19355240bc2e None admin] RBAC: Authorizing `identity:get_user(user_id=af76035096944c6089b84898f4dfd914)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:14.473090 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bffeed5-c3f7-430b-af61-19355240bc2e None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:14.474679 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 594/1197] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Sep 1 16:23:14 2022] GET /identity/v3/users/af76035096944c6089b84898f4dfd914 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:14.480603 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b9532-0a7f-4587-8d2c-0a285b8f6c5e None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:14.578133 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b9532-0a7f-4587-8d2c-0a285b8f6c5e None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:14.610360 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fad688-1657-48f7-8ce0-09c0f14a1d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:14.610955 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fad688-1657-48f7-8ce0-09c0f14a1d9e None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:14.611238 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fad688-1657-48f7-8ce0-09c0f14a1d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:14.611515 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fad688-1657-48f7-8ce0-09c0f14a1d9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:14.663996 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:14.677077 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fad688-1657-48f7-8ce0-09c0f14a1d9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:14.679074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fad688-1657-48f7-8ce0-09c0f14a1d9e None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:14.773059 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 604/1198] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:23:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:23:14.778835 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bffeed5-c3f7-430b-af61-19355240bc2e None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:14.874522 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bffeed5-c3f7-430b-af61-19355240bc2e None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:14.906428 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a28231-5b58-4afa-8963-8999191e44b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:14.907033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a28231-5b58-4afa-8963-8999191e44b8 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:14.907319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a28231-5b58-4afa-8963-8999191e44b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:14.907588 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a28231-5b58-4afa-8963-8999191e44b8 None admin] PATH_INFO: `/v3/projects/06db84a05016481382ec1d9400c953b9` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:14.908670 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a28231-5b58-4afa-8963-8999191e44b8 None admin] RBAC: Authorizing `identity:get_project(project_id=06db84a05016481382ec1d9400c953b9)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:14.910065 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a28231-5b58-4afa-8963-8999191e44b8 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:14.911656 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 595/1199] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Sep 1 16:23:14 2022] GET /identity/v3/projects/06db84a05016481382ec1d9400c953b9 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:14.917467 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fad688-1657-48f7-8ce0-09c0f14a1d9e None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:14.999174 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:15.014455 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fad688-1657-48f7-8ce0-09c0f14a1d9e None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:15.047341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d25d3-3c3a-4d41-9489-8f8fc28c0594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:15.047934 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d25d3-3c3a-4d41-9489-8f8fc28c0594 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:15.048218 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d25d3-3c3a-4d41-9489-8f8fc28c0594 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:15.048492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d25d3-3c3a-4d41-9489-8f8fc28c0594 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:15.049209 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177d25d3-3c3a-4d41-9489-8f8fc28c0594 None admin] RBAC: Adding query filter params (role.id=e8ba063905484d22993bcf04830091a3, scope.project.id=06db84a05016481382ec1d9400c953b9, user.id=af76035096944c6089b84898f4dfd914) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:15.049520 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177d25d3-3c3a-4d41-9489-8f8fc28c0594 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:15.050959 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177d25d3-3c3a-4d41-9489-8f8fc28c0594 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:15.057110 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 605/1200] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:23:14 2022] GET /identity/v3/role_assignments?user.id=af76035096944c6089b84898f4dfd914&scope.project.id=06db84a05016481382ec1d9400c953b9&role.id=e8ba063905484d22993bcf04830091a3 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:24.902506 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:25.054575 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=25 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:11.653854 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ae518-f8aa-4779-be53-0f58b6ca7af0 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:11.654472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ae518-f8aa-4779-be53-0f58b6ca7af0 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:11.654472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ae518-f8aa-4779-be53-0f58b6ca7af0 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:11.656480 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 596/1201] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:11.665821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ba395-00e5-4373-91e8-11d3859d6d73 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:11.665821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ba395-00e5-4373-91e8-11d3859d6d73 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:11.665821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ba395-00e5-4373-91e8-11d3859d6d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:11.687230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-720ba395-00e5-4373-91e8-11d3859d6d73 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:11.688888 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-720ba395-00e5-4373-91e8-11d3859d6d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:11.780624 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 606/1202] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:12.095404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ae518-f8aa-4779-be53-0f58b6ca7af0 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:12.245185 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ae518-f8aa-4779-be53-0f58b6ca7af0 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:12.294829 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aadf58b-c4c4-40b2-90ba-df489144cd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:12.295847 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aadf58b-c4c4-40b2-90ba-df489144cd4c None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:12.296327 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aadf58b-c4c4-40b2-90ba-df489144cd4c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:12.296625 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aadf58b-c4c4-40b2-90ba-df489144cd4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:12.310444 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aadf58b-c4c4-40b2-90ba-df489144cd4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:12.312335 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aadf58b-c4c4-40b2-90ba-df489144cd4c None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:12.332855 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-1aadf58b-c4c4-40b2-90ba-df489144cd4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:12.332855 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:12.332855 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:12.332855 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:12.332855 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:12.332855 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:12.332855 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:12.332855 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:12.332855 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:12.332855 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:12.332855 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:12.332855 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:12.332855 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:12.332855 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:12.332855 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:12.332855 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:12.333973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:12.333973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:12.333973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:12.333973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:12.333973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:12.333973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:12.333973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:12.333973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:12.333973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:12.333973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:12.333973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:12.333973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:12.333973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:12.333973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:12.333973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:12.333973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:12.333973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:12.334985 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:12.334985 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:12.334985 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:24:12.334985 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 597/1203] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:12.340588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ba395-00e5-4373-91e8-11d3859d6d73 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:12.492900 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ba395-00e5-4373-91e8-11d3859d6d73 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:12.531398 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:12.539095 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7cb8b8-4894-4491-a486-0372581b1a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:12.539821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7cb8b8-4894-4491-a486-0372581b1a41 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:12.540185 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7cb8b8-4894-4491-a486-0372581b1a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:12.540541 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7cb8b8-4894-4491-a486-0372581b1a41 None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:12.541349 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7cb8b8-4894-4491-a486-0372581b1a41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:12.541696 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7cb8b8-4894-4491-a486-0372581b1a41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:12.543232 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7cb8b8-4894-4491-a486-0372581b1a41 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:12.562209 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 607/1204] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:12.563508 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aadf58b-c4c4-40b2-90ba-df489144cd4c None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:12.632922 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:12.689772 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aadf58b-c4c4-40b2-90ba-df489144cd4c None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:12.729778 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed9d190-f16f-43dc-b811-e4e8009a8117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:12.730486 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9d190-f16f-43dc-b811-e4e8009a8117 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:12.730824 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9d190-f16f-43dc-b811-e4e8009a8117 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:12.731101 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9d190-f16f-43dc-b811-e4e8009a8117 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:12.731750 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed9d190-f16f-43dc-b811-e4e8009a8117 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:12.734989 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed9d190-f16f-43dc-b811-e4e8009a8117 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:12.762272 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 598/1205] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Sep 1 16:24:12 2022] POST /identity/v3/users => generated 251 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:24:14.908222 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77297837-35bb-48f5-aea4-166e79269566 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:14.908825 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77297837-35bb-48f5-aea4-166e79269566 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:14.908825 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77297837-35bb-48f5-aea4-166e79269566 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:14.910119 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 608/1206] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:14.916087 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19acddb-e286-4c1b-a7bb-9ebe0df2336e None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:14.916541 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19acddb-e286-4c1b-a7bb-9ebe0df2336e None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:14.916739 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19acddb-e286-4c1b-a7bb-9ebe0df2336e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:14.940208 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-d19acddb-e286-4c1b-a7bb-9ebe0df2336e None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:14.940731 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d19acddb-e286-4c1b-a7bb-9ebe0df2336e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:15.020929 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 599/1207] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:15.311239 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77297837-35bb-48f5-aea4-166e79269566 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:15.451632 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77297837-35bb-48f5-aea4-166e79269566 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:15.492359 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbad6a1-a198-48fa-9b98-08128c119f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:15.493067 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbad6a1-a198-48fa-9b98-08128c119f18 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:15.493356 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbad6a1-a198-48fa-9b98-08128c119f18 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:15.493640 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbad6a1-a198-48fa-9b98-08128c119f18 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:15.505305 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbad6a1-a198-48fa-9b98-08128c119f18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:24:15.511389 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-ccbad6a1-a198-48fa-9b98-08128c119f18 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:24:15.511389 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:15.511389 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:15.511389 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:15.511389 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:15.511389 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:15.511389 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:15.511389 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:15.511389 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:15.511389 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:15.511389 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:15.511389 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:15.511389 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:24:15.511389 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:24:15.511389 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:24:15.511389 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:24:15.512617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:24:15.512617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:24:15.512617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:24:15.512617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:24:15.512617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:15.512617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:15.512617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:15.512617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:15.512617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:15.512617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:15.512617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:15.512617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:15.512617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:15.512617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:15.512617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:15.512617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:15.513598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:15.513598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:15.513598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:15.513598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:15.513598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:24:15.513598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:24:15.513598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:24:15.513598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:24:15.513598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:24:15.513598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:24:15.513598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:24:15.513598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:24:15.513598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 609/1208] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 1 16:24:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:15.519183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19acddb-e286-4c1b-a7bb-9ebe0df2336e None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:15.562167 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:15.640935 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19acddb-e286-4c1b-a7bb-9ebe0df2336e None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:15.684125 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113844cd-6aed-445d-ae15-894d780a8769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:15.684893 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113844cd-6aed-445d-ae15-894d780a8769 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:15.685182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113844cd-6aed-445d-ae15-894d780a8769 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:15.685461 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113844cd-6aed-445d-ae15-894d780a8769 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:15.686345 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113844cd-6aed-445d-ae15-894d780a8769 None admin] RBAC: Adding query filter params (name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:15.686695 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113844cd-6aed-445d-ae15-894d780a8769 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:15.688402 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113844cd-6aed-445d-ae15-894d780a8769 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:15.700261 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 600/1209] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 1 16:24:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:15.706356 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbad6a1-a198-48fa-9b98-08128c119f18 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:15.780703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:15.859777 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbad6a1-a198-48fa-9b98-08128c119f18 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:15.906159 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a3e46-7d53-483a-88bd-dbd8e0c3c765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:15.907017 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a3e46-7d53-483a-88bd-dbd8e0c3c765 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:15.907305 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a3e46-7d53-483a-88bd-dbd8e0c3c765 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:15.907616 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a3e46-7d53-483a-88bd-dbd8e0c3c765 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:15.920744 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a3e46-7d53-483a-88bd-dbd8e0c3c765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:15.922515 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a3e46-7d53-483a-88bd-dbd8e0c3c765 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:15.941264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-2f5a3e46-7d53-483a-88bd-dbd8e0c3c765 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:15.941264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:15.941264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:15.941264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:15.941264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:15.941264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:15.941264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:15.941264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:15.941264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:15.941264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:15.941264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:15.941264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:15.941264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:15.941264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:15.941264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:15.941264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:15.942492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:15.942492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:15.942492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:15.942492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:15.942492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:15.942492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:15.942492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:15.942492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:15.942492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:15.942492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:15.942492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:15.942492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:15.942492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:15.942492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:15.942492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:15.942492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:15.942492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:15.943511 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:15.943511 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:15.943511 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:24:15.943511 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 610/1210] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:24:15.948954 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113844cd-6aed-445d-ae15-894d780a8769 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:16.064852 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:16.067759 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113844cd-6aed-445d-ae15-894d780a8769 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:16.107429 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec90af4-bbbe-459b-b82c-782ffa3961dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:16.108207 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec90af4-bbbe-459b-b82c-782ffa3961dd None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:16.108532 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec90af4-bbbe-459b-b82c-782ffa3961dd None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:16.108823 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec90af4-bbbe-459b-b82c-782ffa3961dd None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:16.109655 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec90af4-bbbe-459b-b82c-782ffa3961dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:16.110142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec90af4-bbbe-459b-b82c-782ffa3961dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:16.111835 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec90af4-bbbe-459b-b82c-782ffa3961dd None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:16.132677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 601/1211] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:16.139035 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a3e46-7d53-483a-88bd-dbd8e0c3c765 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:16.257485 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a3e46-7d53-483a-88bd-dbd8e0c3c765 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:16.259263 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:16.297923 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a37ce60-0024-424a-ab44-f81205ec3933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:16.300405 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a37ce60-0024-424a-ab44-f81205ec3933 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:16.300405 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a37ce60-0024-424a-ab44-f81205ec3933 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:16.300405 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a37ce60-0024-424a-ab44-f81205ec3933 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:16.308297 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a37ce60-0024-424a-ab44-f81205ec3933 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:16.310201 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a37ce60-0024-424a-ab44-f81205ec3933 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:16.325223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-0a37ce60-0024-424a-ab44-f81205ec3933 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:24:16.325223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:16.325223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:16.325223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:16.325223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:16.325223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:16.325223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:16.325223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:16.325223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:16.325223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:16.325223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:16.325223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:16.325223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:24:16.325223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:24:16.325223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:24:16.325223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:24:16.326487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:16.326487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:16.326487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:24:16.326487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:16.326487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:24:16.326487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:16.326487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:16.326487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:16.326487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:16.326487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:16.326487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:16.326487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:16.326487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:16.326487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:16.326487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:16.326487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:16.326487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:16.326487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:16.327441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:16.327441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:16.327441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:24:16.327441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:24:16.327441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:24:16.327441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:24:16.327441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:24:16.327441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:24:16.327441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:24:16.327441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:24:16.327441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 611/1212] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:24:16.332841 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec90af4-bbbe-459b-b82c-782ffa3961dd None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:16.453818 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec90af4-bbbe-459b-b82c-782ffa3961dd None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:16.497846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18810f78-bd22-430d-943a-6254d95e0ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:16.498676 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18810f78-bd22-430d-943a-6254d95e0ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:16.498978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18810f78-bd22-430d-943a-6254d95e0ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:16.499250 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18810f78-bd22-430d-943a-6254d95e0ed5 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:16.500173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18810f78-bd22-430d-943a-6254d95e0ed5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:16.500523 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18810f78-bd22-430d-943a-6254d95e0ed5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:16.502312 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18810f78-bd22-430d-943a-6254d95e0ed5 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:16.526313 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 602/1213] 10.222.0.18 () {56 vars in 1143 bytes} [Thu Sep 1 16:24:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:16.533094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a37ce60-0024-424a-ab44-f81205ec3933 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:16.652864 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a37ce60-0024-424a-ab44-f81205ec3933 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:16.699698 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8672a-aecb-4085-8fdb-229aaed62d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:16.700471 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8672a-aecb-4085-8fdb-229aaed62d0b None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:16.700853 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8672a-aecb-4085-8fdb-229aaed62d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:16.701173 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8672a-aecb-4085-8fdb-229aaed62d0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:16.743404 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:16.791247 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8672a-aecb-4085-8fdb-229aaed62d0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:16.792920 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8672a-aecb-4085-8fdb-229aaed62d0b None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:16.918677 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 612/1214] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:24:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:24:16.924894 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18810f78-bd22-430d-943a-6254d95e0ed5 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:16.946833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:17.037093 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18810f78-bd22-430d-943a-6254d95e0ed5 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:17.073755 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d4d22-e38c-40c9-a93f-fbbcc1b98a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:17.074698 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d4d22-e38c-40c9-a93f-fbbcc1b98a54 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:17.075144 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d4d22-e38c-40c9-a93f-fbbcc1b98a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:17.075554 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d4d22-e38c-40c9-a93f-fbbcc1b98a54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:17.088386 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1d4d22-e38c-40c9-a93f-fbbcc1b98a54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:17.090007 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1d4d22-e38c-40c9-a93f-fbbcc1b98a54 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:17.105210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-9c1d4d22-e38c-40c9-a93f-fbbcc1b98a54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:17.105210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:17.105210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:17.105210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:17.105210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:17.105210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:17.105210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:17.105210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:17.105210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:17.105210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:17.105210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:17.105210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:17.105210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:17.105210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:17.105210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:17.105210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:17.106387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:17.106387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:17.106387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:17.106387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:17.106387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:17.106387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:17.106387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:17.106387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:17.106387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:17.106387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:17.106387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:17.106387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:17.106387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:17.106387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:17.106387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:17.106387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:17.106387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:17.107489 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:17.107489 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:17.107489 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:24:17.113827 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 603/1215] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:17.116041 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8672a-aecb-4085-8fdb-229aaed62d0b None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:17.194270 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:17.243540 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8672a-aecb-4085-8fdb-229aaed62d0b None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:17.287066 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab48585-2156-4a25-927f-8c3168ea5a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:17.287902 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab48585-2156-4a25-927f-8c3168ea5a99 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:17.288289 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab48585-2156-4a25-927f-8c3168ea5a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:17.288754 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab48585-2156-4a25-927f-8c3168ea5a99 None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:17.289557 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab48585-2156-4a25-927f-8c3168ea5a99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:17.290000 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab48585-2156-4a25-927f-8c3168ea5a99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:17.292028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab48585-2156-4a25-927f-8c3168ea5a99 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:17.306928 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 613/1216] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:17.312480 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d4d22-e38c-40c9-a93f-fbbcc1b98a54 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:17.401692 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:17.437647 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d4d22-e38c-40c9-a93f-fbbcc1b98a54 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:17.475414 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b36be-ed4c-4934-becd-b8675fc8d185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:17.476025 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b36be-ed4c-4934-becd-b8675fc8d185 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:17.476325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b36be-ed4c-4934-becd-b8675fc8d185 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:17.476628 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b36be-ed4c-4934-becd-b8675fc8d185 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:17.484175 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4b36be-ed4c-4934-becd-b8675fc8d185 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:17.485598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4b36be-ed4c-4934-becd-b8675fc8d185 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:17.497467 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-ad4b36be-ed4c-4934-becd-b8675fc8d185 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:17.497467 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:17.497467 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:17.497467 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:17.497467 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:17.497467 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:17.497467 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:17.497467 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:17.497467 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:17.497467 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:17.497467 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:17.497467 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:17.497467 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:24:17.497467 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:24:17.497467 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:24:17.498679 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:24:17.498679 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:17.498679 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:17.498679 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:17.498679 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:17.498679 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:17.498679 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:17.498679 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:17.498679 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:17.498679 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:17.498679 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:17.498679 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:17.498679 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:17.498679 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:17.498679 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:17.498679 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:17.498679 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:17.499612 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:24:17.499612 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:24:17.499612 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:24:17.499612 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:24:17.499612 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:24:17.499612 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:24:17.499612 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:17.499612 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:24:17.502585 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 604/1217] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:24:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:24:17.505872 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab48585-2156-4a25-927f-8c3168ea5a99 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:17.620978 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab48585-2156-4a25-927f-8c3168ea5a99 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:17.654887 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:17.669618 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1b39e-5beb-4327-b814-0abba1ada82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:17.670298 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1b39e-5beb-4327-b814-0abba1ada82d None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:17.670597 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1b39e-5beb-4327-b814-0abba1ada82d None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:17.670874 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1b39e-5beb-4327-b814-0abba1ada82d None admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:17.671654 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1b39e-5beb-4327-b814-0abba1ada82d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:17.672006 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1b39e-5beb-4327-b814-0abba1ada82d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:17.673528 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1b39e-5beb-4327-b814-0abba1ada82d None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:17.687905 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 614/1218] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:24:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:17.695142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b36be-ed4c-4934-becd-b8675fc8d185 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:17.818585 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b36be-ed4c-4934-becd-b8675fc8d185 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:17.854531 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:17.862794 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae67373-86a3-4539-81a8-c76e9017ba7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:17.863457 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae67373-86a3-4539-81a8-c76e9017ba7d None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:17.864227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae67373-86a3-4539-81a8-c76e9017ba7d None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:17.864763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae67373-86a3-4539-81a8-c76e9017ba7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:17.865485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae67373-86a3-4539-81a8-c76e9017ba7d None admin] RBAC: Adding query filter params (role.id=b6727ed962fa41109b0a2d6851551f49, scope.project.id=36ce80c4e0d64562a9ab3961be76fad9, user.id=3ddfb151bbf844efae73ec1990e7eaa3) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:17.865790 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae67373-86a3-4539-81a8-c76e9017ba7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:17.867365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae67373-86a3-4539-81a8-c76e9017ba7d None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:17.879447 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 605/1219] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:24:17 2022] GET /identity/v3/role_assignments?user.id=3ddfb151bbf844efae73ec1990e7eaa3&scope.project.id=36ce80c4e0d64562a9ab3961be76fad9&role.id=b6727ed962fa41109b0a2d6851551f49 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:19.904687 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02215c87-0b0d-40b9-9c31-851e3820806a None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:19.905228 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02215c87-0b0d-40b9-9c31-851e3820806a None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:19.905228 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02215c87-0b0d-40b9-9c31-851e3820806a None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:19.906622 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 615/1220] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:19.912048 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70553a9-5477-4de1-a42c-2b937d232147 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:19.912482 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70553a9-5477-4de1-a42c-2b937d232147 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:19.912587 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70553a9-5477-4de1-a42c-2b937d232147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:19.931577 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-f70553a9-5477-4de1-a42c-2b937d232147 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:19.932776 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f70553a9-5477-4de1-a42c-2b937d232147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:20.011169 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 606/1221] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:24:20.283389 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02215c87-0b0d-40b9-9c31-851e3820806a None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:20.394074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02215c87-0b0d-40b9-9c31-851e3820806a None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:20.428006 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047345f9-2993-49a4-84dc-76d65c00f554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:20.428642 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047345f9-2993-49a4-84dc-76d65c00f554 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:20.428924 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047345f9-2993-49a4-84dc-76d65c00f554 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:20.429200 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047345f9-2993-49a4-84dc-76d65c00f554 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:20.443766 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047345f9-2993-49a4-84dc-76d65c00f554 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:24:20.447585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-047345f9-2993-49a4-84dc-76d65c00f554 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:24:20.447585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:20.447585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:20.447585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:20.447585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:20.447585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:20.447585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:20.447585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:20.447585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:20.447585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:20.447585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:20.447585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:20.447585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:24:20.447585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:24:20.447585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:24:20.447585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:24:20.449430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:24:20.449430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:24:20.449430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:24:20.449430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:24:20.449430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:20.449430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:20.449430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:20.449430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:20.449430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:20.449430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:20.449430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:20.449430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:20.449430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:20.449430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:20.449430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:20.449430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:20.451198 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:20.451198 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:20.451198 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:20.451198 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:20.451198 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:24:20.451198 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:24:20.451198 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:24:20.451198 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:24:20.451198 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:24:20.451198 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:24:20.451198 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:24:20.451198 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:24:20.451198 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 616/1222] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 1 16:24:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:20.454718 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70553a9-5477-4de1-a42c-2b937d232147 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:20.561613 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70553a9-5477-4de1-a42c-2b937d232147 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:20.589027 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:20.600352 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae12d35-8a9a-43c6-888d-2d7b79270c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:20.601015 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae12d35-8a9a-43c6-888d-2d7b79270c23 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:20.601300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae12d35-8a9a-43c6-888d-2d7b79270c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:20.601575 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae12d35-8a9a-43c6-888d-2d7b79270c23 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:20.602360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae12d35-8a9a-43c6-888d-2d7b79270c23 None admin] RBAC: Adding query filter params (name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:20.602668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae12d35-8a9a-43c6-888d-2d7b79270c23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:20.604146 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae12d35-8a9a-43c6-888d-2d7b79270c23 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:20.619474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 607/1223] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 1 16:24:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:20.620905 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047345f9-2993-49a4-84dc-76d65c00f554 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:20.668685 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:20.732042 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047345f9-2993-49a4-84dc-76d65c00f554 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:20.767751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d068a44-ef53-460c-bb85-04ee3e278ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:20.768386 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d068a44-ef53-460c-bb85-04ee3e278ec5 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:20.768679 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d068a44-ef53-460c-bb85-04ee3e278ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:20.768938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d068a44-ef53-460c-bb85-04ee3e278ec5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:20.780622 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d068a44-ef53-460c-bb85-04ee3e278ec5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:20.782614 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d068a44-ef53-460c-bb85-04ee3e278ec5 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:20.804197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-4d068a44-ef53-460c-bb85-04ee3e278ec5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:20.804197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:20.804197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:20.804197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:20.804197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:20.804197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:20.804197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:20.804197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:20.804197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:20.804197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:20.804197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:20.804197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:20.804197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:20.804197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:20.804197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:20.804197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:20.805740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:20.805740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:20.805740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:20.805740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:20.805740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:20.805740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:20.805740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:20.805740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:20.805740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:20.805740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:20.805740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:20.805740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:20.805740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:20.805740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:20.805740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:20.805740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:20.805740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:20.806859 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:20.806859 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:20.806859 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:24:20.807535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 617/1224] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:20.813584 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae12d35-8a9a-43c6-888d-2d7b79270c23 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:20.923718 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae12d35-8a9a-43c6-888d-2d7b79270c23 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:20.958856 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248cad70-dae5-4e76-bfc0-d7c6fc0b06c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:20.959474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248cad70-dae5-4e76-bfc0-d7c6fc0b06c2 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:20.959760 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248cad70-dae5-4e76-bfc0-d7c6fc0b06c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:20.960034 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248cad70-dae5-4e76-bfc0-d7c6fc0b06c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:20.960678 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248cad70-dae5-4e76-bfc0-d7c6fc0b06c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:20.960989 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248cad70-dae5-4e76-bfc0-d7c6fc0b06c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:20.962478 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248cad70-dae5-4e76-bfc0-d7c6fc0b06c2 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:20.975550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 608/1225] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:20.981753 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d068a44-ef53-460c-bb85-04ee3e278ec5 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:21.092120 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:21.099572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d068a44-ef53-460c-bb85-04ee3e278ec5 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:21.135084 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecb657-ad7f-4b75-8f66-e1980cb6950b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:21.135846 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecb657-ad7f-4b75-8f66-e1980cb6950b None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:21.136132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecb657-ad7f-4b75-8f66-e1980cb6950b None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:21.136498 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecb657-ad7f-4b75-8f66-e1980cb6950b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:21.145129 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ecb657-ad7f-4b75-8f66-e1980cb6950b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:21.146680 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ecb657-ad7f-4b75-8f66-e1980cb6950b None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:21.158831 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-c6ecb657-ad7f-4b75-8f66-e1980cb6950b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:24:21.158831 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:21.158831 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:21.158831 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:21.158831 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:21.158831 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:21.158831 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:21.158831 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:21.158831 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:21.158831 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:21.158831 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:21.158831 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:21.158831 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:24:21.158831 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:24:21.158831 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:24:21.158831 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:24:21.159997 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:21.159997 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:21.159997 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:24:21.159997 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:21.159997 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:24:21.159997 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:21.159997 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:21.159997 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:21.159997 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:21.159997 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:21.159997 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:21.159997 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:21.159997 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:21.159997 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:21.159997 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:21.159997 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:21.159997 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:21.159997 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:21.161675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:21.161675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:21.161675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:24:21.161675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:24:21.161675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:24:21.161675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:24:21.161675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:24:21.161675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:24:21.161675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:24:21.161675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:24:21.162735 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 618/1226] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:21.169303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248cad70-dae5-4e76-bfc0-d7c6fc0b06c2 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:21.225490 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:21.288785 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248cad70-dae5-4e76-bfc0-d7c6fc0b06c2 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:21.325477 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db90b3-fe89-4476-8f90-999183e638c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:21.326120 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db90b3-fe89-4476-8f90-999183e638c2 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:21.326481 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db90b3-fe89-4476-8f90-999183e638c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:21.326784 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db90b3-fe89-4476-8f90-999183e638c2 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:21.327581 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db90b3-fe89-4476-8f90-999183e638c2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:21.327925 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db90b3-fe89-4476-8f90-999183e638c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:21.329351 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db90b3-fe89-4476-8f90-999183e638c2 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:21.354350 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 609/1227] 10.222.0.18 () {56 vars in 1143 bytes} [Thu Sep 1 16:24:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:21.355565 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecb657-ad7f-4b75-8f66-e1980cb6950b None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:21.467088 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecb657-ad7f-4b75-8f66-e1980cb6950b None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:21.505165 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b1519-478b-4f0f-9cb1-644bbe552e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:21.505797 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b1519-478b-4f0f-9cb1-644bbe552e17 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:21.506080 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b1519-478b-4f0f-9cb1-644bbe552e17 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:21.506381 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b1519-478b-4f0f-9cb1-644bbe552e17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:21.519626 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b1519-478b-4f0f-9cb1-644bbe552e17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:21.521533 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b1519-478b-4f0f-9cb1-644bbe552e17 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:21.538314 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-568b1519-478b-4f0f-9cb1-644bbe552e17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:21.538314 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:21.538314 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:21.538314 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:21.538314 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:21.538314 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:21.538314 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:21.538314 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:21.538314 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:21.538314 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:21.538314 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:21.538314 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:21.538314 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:21.538314 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:21.538314 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:21.538314 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:21.539937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:21.539937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:21.539937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:21.539937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:21.539937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:21.539937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:21.539937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:21.539937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:21.539937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:21.539937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:21.539937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:21.539937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:21.539937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:21.539937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:21.539937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:21.539937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:21.539937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:21.541095 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:21.541095 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:21.541095 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:24:21.541095 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 619/1228] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:21.546279 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db90b3-fe89-4476-8f90-999183e638c2 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:21.658668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db90b3-fe89-4476-8f90-999183e638c2 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:21.661619 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:21.696452 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f36ac-958a-44b4-b469-3df7258fe792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:21.697608 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f36ac-958a-44b4-b469-3df7258fe792 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:21.698074 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f36ac-958a-44b4-b469-3df7258fe792 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:21.698387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f36ac-958a-44b4-b469-3df7258fe792 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:21.699202 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f36ac-958a-44b4-b469-3df7258fe792 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:21.699657 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f36ac-958a-44b4-b469-3df7258fe792 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:21.701349 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f36ac-958a-44b4-b469-3df7258fe792 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:21.715709 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 610/1229] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:21.721561 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b1519-478b-4f0f-9cb1-644bbe552e17 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:21.731832 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:21.836025 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b1519-478b-4f0f-9cb1-644bbe552e17 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:21.872018 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07d6f5-a51f-473c-9e9d-c0b789f5cfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:21.872715 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07d6f5-a51f-473c-9e9d-c0b789f5cfcd None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:21.872993 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07d6f5-a51f-473c-9e9d-c0b789f5cfcd None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:21.873313 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07d6f5-a51f-473c-9e9d-c0b789f5cfcd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:21.881094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b07d6f5-a51f-473c-9e9d-c0b789f5cfcd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:21.882747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b07d6f5-a51f-473c-9e9d-c0b789f5cfcd None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:21.899268 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-6b07d6f5-a51f-473c-9e9d-c0b789f5cfcd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:21.899268 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:21.899268 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:21.899268 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:21.899268 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:21.899268 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:21.899268 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:21.899268 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:21.899268 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:21.899268 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:21.899268 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:21.899268 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:21.899268 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:24:21.899268 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:24:21.899268 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:24:21.900366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:24:21.900366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:21.900366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:21.900366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:21.900366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:21.900366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:21.900366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:21.900366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:21.900366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:21.900366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:21.900366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:21.900366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:21.900366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:21.900366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:21.900366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:21.900366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:21.900366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:21.901386 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:24:21.901386 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:24:21.901386 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:24:21.901386 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:24:21.901386 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:24:21.901386 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:24:21.901386 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:21.901386 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:24:21.901386 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 620/1230] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:24:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:21.906785 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f36ac-958a-44b4-b469-3df7258fe792 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:22.015872 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f36ac-958a-44b4-b469-3df7258fe792 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:22.057019 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e65abc-d626-4802-8c83-bc1fe0a4fc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:22.057639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e65abc-d626-4802-8c83-bc1fe0a4fc60 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:22.057920 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e65abc-d626-4802-8c83-bc1fe0a4fc60 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:22.058221 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e65abc-d626-4802-8c83-bc1fe0a4fc60 None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:22.058918 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e65abc-d626-4802-8c83-bc1fe0a4fc60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:22.059227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e65abc-d626-4802-8c83-bc1fe0a4fc60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:22.060652 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e65abc-d626-4802-8c83-bc1fe0a4fc60 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:22.073129 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 611/1231] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:24:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:22.079494 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07d6f5-a51f-473c-9e9d-c0b789f5cfcd None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:22.156519 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:22.200149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07d6f5-a51f-473c-9e9d-c0b789f5cfcd None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:22.238846 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0dfbea-98a4-4333-b48f-4e9fe27d0e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:22.239935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0dfbea-98a4-4333-b48f-4e9fe27d0e20 None admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:22.240343 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0dfbea-98a4-4333-b48f-4e9fe27d0e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:22.240653 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0dfbea-98a4-4333-b48f-4e9fe27d0e20 None admin] PATH_INFO: `/v3/projects/36ce80c4e0d64562a9ab3961be76fad9/users/3ddfb151bbf844efae73ec1990e7eaa3/roles/b6727ed962fa41109b0a2d6851551f49` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:22.258341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0dfbea-98a4-4333-b48f-4e9fe27d0e20 None admin] RBAC: Authorizing `identity:create_grant(project_id=36ce80c4e0d64562a9ab3961be76fad9, user_id=3ddfb151bbf844efae73ec1990e7eaa3, role_id=b6727ed962fa41109b0a2d6851551f49)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:22.259923 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0dfbea-98a4-4333-b48f-4e9fe27d0e20 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:22.272852 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 621/1232] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 1 16:24:22 2022] PUT /identity/v3/projects/36ce80c4e0d64562a9ab3961be76fad9/users/3ddfb151bbf844efae73ec1990e7eaa3/roles/b6727ed962fa41109b0a2d6851551f49 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:24:24.188406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b4d65-6eb4-453f-8d42-0a716b07e5e4 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:24.188976 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b4d65-6eb4-453f-8d42-0a716b07e5e4 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:24.188976 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b4d65-6eb4-453f-8d42-0a716b07e5e4 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:24.190378 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 612/1233] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:24:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:24:24.195722 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ac20b-4c07-4de4-bff4-6ac6ce8e3e95 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:24.196155 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ac20b-4c07-4de4-bff4-6ac6ce8e3e95 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:24.196262 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ac20b-4c07-4de4-bff4-6ac6ce8e3e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:24.215048 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-c31ac20b-4c07-4de4-bff4-6ac6ce8e3e95 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:24.216240 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-c31ac20b-4c07-4de4-bff4-6ac6ce8e3e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:24.287242 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 622/1234] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:24.557503 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b4d65-6eb4-453f-8d42-0a716b07e5e4 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:24.589041 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:24.671423 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b4d65-6eb4-453f-8d42-0a716b07e5e4 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:24.710755 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d95ebd-8c61-4296-b395-1767bffd4208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:24.711389 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d95ebd-8c61-4296-b395-1767bffd4208 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:24.711799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d95ebd-8c61-4296-b395-1767bffd4208 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:24.712205 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d95ebd-8c61-4296-b395-1767bffd4208 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:24.723357 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d95ebd-8c61-4296-b395-1767bffd4208 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:24:24.728157 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-86d95ebd-8c61-4296-b395-1767bffd4208 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:24:24.728157 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:24.728157 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:24.728157 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:24.728157 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:24.728157 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:24.728157 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:24.728157 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:24.728157 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:24.728157 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:24.728157 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:24.728157 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:24.728157 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:24:24.728157 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:24:24.728157 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:24:24.728157 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:24:24.729314 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:24:24.729314 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:24:24.729314 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:24:24.729314 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:24:24.729314 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:24.729314 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:24.729314 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:24.729314 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:24.729314 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:24.729314 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:24.729314 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:24.729314 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:24.729314 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:24.729314 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:24.729314 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:24.729314 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:24.730388 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:24.730388 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:24.730388 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:24.730388 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:24.730388 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:24:24.730388 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:24:24.730388 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:24:24.730388 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:24:24.730388 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:24:24.730388 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:24:24.730388 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:24:24.730388 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:24:24.730388 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 613/1235] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Sep 1 16:24:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:24:24.735817 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ac20b-4c07-4de4-bff4-6ac6ce8e3e95 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:24.784968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:24.842847 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ac20b-4c07-4de4-bff4-6ac6ce8e3e95 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:24.879827 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976b761-f2a2-4ff0-9e2a-fa1f0e08426c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:24.880451 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976b761-f2a2-4ff0-9e2a-fa1f0e08426c None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:24.880736 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976b761-f2a2-4ff0-9e2a-fa1f0e08426c None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:24.881015 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976b761-f2a2-4ff0-9e2a-fa1f0e08426c None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:24.881695 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976b761-f2a2-4ff0-9e2a-fa1f0e08426c None admin] RBAC: Adding query filter params (name=service) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:24.882005 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976b761-f2a2-4ff0-9e2a-fa1f0e08426c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:24.883520 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976b761-f2a2-4ff0-9e2a-fa1f0e08426c None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:24.897422 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 623/1236] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Sep 1 16:24:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:24.899306 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d95ebd-8c61-4296-b395-1767bffd4208 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:25.001861 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:25.019749 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d95ebd-8c61-4296-b395-1767bffd4208 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:25.056207 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a0e80-3ea5-4b82-b58f-1978abc29af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:25.056847 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a0e80-3ea5-4b82-b58f-1978abc29af6 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:25.057131 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a0e80-3ea5-4b82-b58f-1978abc29af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:25.057428 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a0e80-3ea5-4b82-b58f-1978abc29af6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:25.068030 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a0e80-3ea5-4b82-b58f-1978abc29af6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:25.069466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a0e80-3ea5-4b82-b58f-1978abc29af6 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:25.084453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-243a0e80-3ea5-4b82-b58f-1978abc29af6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:25.084453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:25.084453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:25.084453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:25.084453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:25.084453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:25.084453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:25.084453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:25.084453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:25.084453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:25.084453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:25.084453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:25.084453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:25.084453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:25.084453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:25.084453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:25.085550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:25.085550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:25.085550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:25.085550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:25.085550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:25.085550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:25.085550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:25.085550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:25.085550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:25.085550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:25.085550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:25.085550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:25.085550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:25.085550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:25.085550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:25.085550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:25.085550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:25.086507 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:25.086507 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:25.086507 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:24:25.086507 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 614/1237] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:25.091457 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976b761-f2a2-4ff0-9e2a-fa1f0e08426c None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:25.199391 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976b761-f2a2-4ff0-9e2a-fa1f0e08426c None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:25.209344 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:25.234814 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c261b4-0ff0-4555-8e04-750019c3bd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:25.235416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c261b4-0ff0-4555-8e04-750019c3bd0f None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:25.235787 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c261b4-0ff0-4555-8e04-750019c3bd0f None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:25.236010 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c261b4-0ff0-4555-8e04-750019c3bd0f None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:25.236735 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c261b4-0ff0-4555-8e04-750019c3bd0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:25.237078 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c261b4-0ff0-4555-8e04-750019c3bd0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:25.238707 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c261b4-0ff0-4555-8e04-750019c3bd0f None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:25.251393 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 624/1238] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:25.257521 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a0e80-3ea5-4b82-b58f-1978abc29af6 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:25.359284 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a0e80-3ea5-4b82-b58f-1978abc29af6 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:25.395361 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f08533-6d47-405d-b33f-d0d0006449e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:25.396362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f08533-6d47-405d-b33f-d0d0006449e3 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:25.397085 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f08533-6d47-405d-b33f-d0d0006449e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:25.397390 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f08533-6d47-405d-b33f-d0d0006449e3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:25.401078 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:25.407499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f08533-6d47-405d-b33f-d0d0006449e3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:25.409106 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f08533-6d47-405d-b33f-d0d0006449e3 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:25.421677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-f8f08533-6d47-405d-b33f-d0d0006449e3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:24:25.421677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:25.421677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:25.421677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:25.421677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:25.421677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:25.421677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:25.421677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:25.421677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:25.421677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:25.421677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:25.421677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:25.421677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:24:25.421677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:24:25.421677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:24:25.421677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:24:25.422723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:25.422723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:25.422723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:24:25.422723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:25.422723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:24:25.422723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:25.422723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:25.422723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:25.422723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:25.422723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:25.422723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:25.422723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:25.422723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:25.422723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:25.422723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:25.422723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:25.422723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:25.422723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:25.423709 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:25.423709 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:25.423709 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:24:25.423709 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:24:25.423709 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:24:25.423709 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:24:25.423709 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:24:25.423709 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:24:25.423709 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:24:25.423709 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:24:25.423709 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 615/1239] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:25.428809 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c261b4-0ff0-4555-8e04-750019c3bd0f None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:25.543350 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c261b4-0ff0-4555-8e04-750019c3bd0f None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:25.590051 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd05be7-9c73-424d-967b-db0d4a638346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:25.590939 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd05be7-9c73-424d-967b-db0d4a638346 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:25.591224 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd05be7-9c73-424d-967b-db0d4a638346 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:25.591498 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd05be7-9c73-424d-967b-db0d4a638346 None admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:25.592365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd05be7-9c73-424d-967b-db0d4a638346 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:25.592808 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd05be7-9c73-424d-967b-db0d4a638346 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:25.594740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd05be7-9c73-424d-967b-db0d4a638346 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:25.613580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 625/1240] 10.222.0.18 () {56 vars in 1143 bytes} [Thu Sep 1 16:24:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:25.620009 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f08533-6d47-405d-b33f-d0d0006449e3 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:25.723965 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f08533-6d47-405d-b33f-d0d0006449e3 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:25.759484 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089fb400-f77e-4e97-aa54-8723e7bd3fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:25.760145 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fb400-f77e-4e97-aa54-8723e7bd3fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:25.760443 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fb400-f77e-4e97-aa54-8723e7bd3fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:25.760744 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fb400-f77e-4e97-aa54-8723e7bd3fe2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:25.831595 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:25.839037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089fb400-f77e-4e97-aa54-8723e7bd3fe2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:25.840498 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089fb400-f77e-4e97-aa54-8723e7bd3fe2 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:25.967585 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 616/1241] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:24:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:24:25.973740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd05be7-9c73-424d-967b-db0d4a638346 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:26.007276 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:26.087085 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd05be7-9c73-424d-967b-db0d4a638346 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:26.124790 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f6b6c-2d33-4579-9510-f854d0f41835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:26.125474 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f6b6c-2d33-4579-9510-f854d0f41835 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:26.125790 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f6b6c-2d33-4579-9510-f854d0f41835 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:26.126102 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f6b6c-2d33-4579-9510-f854d0f41835 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:26.137683 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3f6b6c-2d33-4579-9510-f854d0f41835 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:26.139211 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3f6b6c-2d33-4579-9510-f854d0f41835 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:26.156137 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-0b3f6b6c-2d33-4579-9510-f854d0f41835 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:26.156137 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:26.156137 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:26.156137 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:26.156137 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:26.156137 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:26.156137 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:26.156137 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:26.156137 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:26.156137 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:26.156137 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:26.156137 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:26.156137 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:26.156137 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:26.156137 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:26.156137 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:26.157315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:26.157315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:26.157315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:26.157315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:26.157315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:26.157315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:26.157315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:26.157315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:26.157315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:26.157315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:26.157315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:26.157315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:26.157315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:26.157315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:26.157315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:26.157315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:26.157315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:26.158464 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:26.158464 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:26.158464 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:24:26.158464 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 626/1242] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:26.163492 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089fb400-f77e-4e97-aa54-8723e7bd3fe2 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:26.269409 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:26.277328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089fb400-f77e-4e97-aa54-8723e7bd3fe2 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:26.313899 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25911919-ba5c-4072-860f-7f960f7b6ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:26.314555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25911919-ba5c-4072-860f-7f960f7b6ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:26.314839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25911919-ba5c-4072-860f-7f960f7b6ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:26.315138 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25911919-ba5c-4072-860f-7f960f7b6ef0 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:26.315912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25911919-ba5c-4072-860f-7f960f7b6ef0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:26.316771 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25911919-ba5c-4072-860f-7f960f7b6ef0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:26.318231 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25911919-ba5c-4072-860f-7f960f7b6ef0 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:26.338468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 617/1243] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:26.340721 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f6b6c-2d33-4579-9510-f854d0f41835 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:26.431206 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:26.452968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f6b6c-2d33-4579-9510-f854d0f41835 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:26.490039 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef298c-6664-42de-bd34-ab873b08bb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:26.490710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef298c-6664-42de-bd34-ab873b08bb2e None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:26.490988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef298c-6664-42de-bd34-ab873b08bb2e None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:26.491527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef298c-6664-42de-bd34-ab873b08bb2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:26.501429 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fef298c-6664-42de-bd34-ab873b08bb2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:26.502936 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fef298c-6664-42de-bd34-ab873b08bb2e None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:26.517046 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-5fef298c-6664-42de-bd34-ab873b08bb2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:26.517046 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:26.517046 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:26.517046 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:26.517046 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:26.517046 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:26.517046 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:26.517046 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:26.517046 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:26.517046 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:26.517046 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:26.517046 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:26.517046 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:24:26.517046 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:24:26.517046 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:24:26.518055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:24:26.518055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:26.518055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:26.518055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:26.518055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:26.518055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:26.518055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:26.518055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:26.518055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:26.518055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:26.518055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:26.518055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:26.518055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:26.518055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:26.518055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:26.518055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:26.518055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:26.519000 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:24:26.519000 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:24:26.519000 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:24:26.519000 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:24:26.519000 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:24:26.519000 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:24:26.519000 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:26.519000 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:24:26.519000 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 627/1244] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:24:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:26.524646 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25911919-ba5c-4072-860f-7f960f7b6ef0 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:26.646452 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25911919-ba5c-4072-860f-7f960f7b6ef0 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:26.683660 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe93f11-5161-4040-b2c1-652c3ddeb7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:26.684305 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe93f11-5161-4040-b2c1-652c3ddeb7b2 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:26.684614 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe93f11-5161-4040-b2c1-652c3ddeb7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:26.684926 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe93f11-5161-4040-b2c1-652c3ddeb7b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:26.685618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe93f11-5161-4040-b2c1-652c3ddeb7b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:26.685931 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe93f11-5161-4040-b2c1-652c3ddeb7b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:26.687419 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe93f11-5161-4040-b2c1-652c3ddeb7b2 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:26.700796 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 618/1245] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:24:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:26.707671 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef298c-6664-42de-bd34-ab873b08bb2e None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:26.815049 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef298c-6664-42de-bd34-ab873b08bb2e None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:26.852834 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9020eb6-1ac9-401b-834f-bb0be1d724d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:26.853540 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9020eb6-1ac9-401b-834f-bb0be1d724d5 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:26.853847 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9020eb6-1ac9-401b-834f-bb0be1d724d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:26.854174 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9020eb6-1ac9-401b-834f-bb0be1d724d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:26.854991 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9020eb6-1ac9-401b-834f-bb0be1d724d5 None admin] RBAC: Adding query filter params (role.id=b6727ed962fa41109b0a2d6851551f49, scope.project.id=36ce80c4e0d64562a9ab3961be76fad9, user.id=3ddfb151bbf844efae73ec1990e7eaa3) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:26.855345 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9020eb6-1ac9-401b-834f-bb0be1d724d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:26.856851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9020eb6-1ac9-401b-834f-bb0be1d724d5 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:26.865024 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 628/1246] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:24:26 2022] GET /identity/v3/role_assignments?user.id=3ddfb151bbf844efae73ec1990e7eaa3&scope.project.id=36ce80c4e0d64562a9ab3961be76fad9&role.id=b6727ed962fa41109b0a2d6851551f49 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:29.096265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a865bbb-940c-4a0c-bc65-f6c09358fe85 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:29.096870 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a865bbb-940c-4a0c-bc65-f6c09358fe85 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:29.096870 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a865bbb-940c-4a0c-bc65-f6c09358fe85 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:29.098677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 619/1247] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:24:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:29.104547 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a0c35b-33e4-45c9-b2ca-4fd692e25ae2 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:29.105075 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a0c35b-33e4-45c9-b2ca-4fd692e25ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:29.105189 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a0c35b-33e4-45c9-b2ca-4fd692e25ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:29.111944 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:29.127641 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-59a0c35b-33e4-45c9-b2ca-4fd692e25ae2 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:29.129147 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-59a0c35b-33e4-45c9-b2ca-4fd692e25ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:29.209645 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 629/1248] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:29.483040 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a865bbb-940c-4a0c-bc65-f6c09358fe85 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:29.504702 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:29.606694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a865bbb-940c-4a0c-bc65-f6c09358fe85 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:29.643645 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ec483-48fe-41ab-a757-aecc31e4b07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:29.644802 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ec483-48fe-41ab-a757-aecc31e4b07b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:29.645092 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ec483-48fe-41ab-a757-aecc31e4b07b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:29.645366 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ec483-48fe-41ab-a757-aecc31e4b07b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:29.657405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667ec483-48fe-41ab-a757-aecc31e4b07b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:24:29.664308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-667ec483-48fe-41ab-a757-aecc31e4b07b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:24:29.664308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:29.664308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:29.664308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:29.664308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:29.664308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:29.664308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:29.664308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:29.664308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:29.664308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:29.664308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:29.664308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:29.664308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:24:29.664308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:24:29.664308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:24:29.664308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:24:29.665512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:24:29.665512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:24:29.665512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:24:29.665512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:24:29.665512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:29.665512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:29.665512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:29.665512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:29.665512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:29.665512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:29.665512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:29.665512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:29.665512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:29.665512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:29.665512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:29.665512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:29.666396 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:29.666396 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:29.666396 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:29.666396 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:29.666396 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:24:29.666396 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:24:29.666396 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:24:29.666396 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:24:29.666396 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:24:29.666396 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:24:29.666396 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:24:29.666396 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:24:29.666396 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 620/1249] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:24:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:29.672150 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a0c35b-33e4-45c9-b2ca-4fd692e25ae2 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:29.787810 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a0c35b-33e4-45c9-b2ca-4fd692e25ae2 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:29.822966 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:29.828708 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e73c7-f23c-4e3f-afa4-91b0b921de3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:29.829461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e73c7-f23c-4e3f-afa4-91b0b921de3b None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:29.829761 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e73c7-f23c-4e3f-afa4-91b0b921de3b None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:29.830110 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e73c7-f23c-4e3f-afa4-91b0b921de3b None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:29.831152 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e73c7-f23c-4e3f-afa4-91b0b921de3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:29.832083 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e73c7-f23c-4e3f-afa4-91b0b921de3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:29.834053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e73c7-f23c-4e3f-afa4-91b0b921de3b None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:29.844294 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 630/1250] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:24:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:29.849884 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ec483-48fe-41ab-a757-aecc31e4b07b None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:29.941877 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:29.967673 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ec483-48fe-41ab-a757-aecc31e4b07b None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:30.009279 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ebf79-9fa8-44f4-a593-627ecaf7e852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:30.009936 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ebf79-9fa8-44f4-a593-627ecaf7e852 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:30.010215 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ebf79-9fa8-44f4-a593-627ecaf7e852 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:30.010591 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ebf79-9fa8-44f4-a593-627ecaf7e852 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:30.021418 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112ebf79-9fa8-44f4-a593-627ecaf7e852 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:30.022907 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112ebf79-9fa8-44f4-a593-627ecaf7e852 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:30.037983 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-112ebf79-9fa8-44f4-a593-627ecaf7e852 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:30.037983 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:30.037983 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:30.037983 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:30.037983 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:30.037983 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:30.037983 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:30.037983 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:30.037983 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:30.037983 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:30.037983 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:30.037983 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:30.037983 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:30.037983 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:30.037983 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:30.037983 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:30.039105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:30.039105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:30.039105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:30.039105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:30.039105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:30.039105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:30.039105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:30.039105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:30.039105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:30.039105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:30.039105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:30.039105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:30.039105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:30.039105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:30.039105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:30.039105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:30.039105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:30.044818 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:30.044818 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:30.044818 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:24:30.044818 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 621/1251] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:30.045094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e73c7-f23c-4e3f-afa4-91b0b921de3b None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:30.156460 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e73c7-f23c-4e3f-afa4-91b0b921de3b None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:30.191995 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d349a-7897-460e-abef-b92f93e0752b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:30.192531 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d349a-7897-460e-abef-b92f93e0752b None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:30.192856 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d349a-7897-460e-abef-b92f93e0752b None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:30.193080 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d349a-7897-460e-abef-b92f93e0752b None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:30.193776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d349a-7897-460e-abef-b92f93e0752b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:30.194114 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d349a-7897-460e-abef-b92f93e0752b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:30.195598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d349a-7897-460e-abef-b92f93e0752b None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:30.209301 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 631/1252] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:30.215084 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ebf79-9fa8-44f4-a593-627ecaf7e852 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:30.323979 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ebf79-9fa8-44f4-a593-627ecaf7e852 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:30.349593 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:30.359434 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b88e0d-772c-4853-9d48-ec25a933051e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:30.360070 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b88e0d-772c-4853-9d48-ec25a933051e None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:30.360352 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b88e0d-772c-4853-9d48-ec25a933051e None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:30.360624 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b88e0d-772c-4853-9d48-ec25a933051e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:30.369743 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b88e0d-772c-4853-9d48-ec25a933051e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:30.371306 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b88e0d-772c-4853-9d48-ec25a933051e None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:30.383969 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-41b88e0d-772c-4853-9d48-ec25a933051e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:24:30.383969 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:30.383969 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:30.383969 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:30.383969 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:30.383969 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:30.383969 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:30.383969 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:30.383969 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:30.383969 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:30.383969 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:30.383969 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:30.383969 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:24:30.383969 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:24:30.383969 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:24:30.383969 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:24:30.385405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:30.385405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:30.385405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:24:30.385405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:30.385405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:24:30.385405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:30.385405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:30.385405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:30.385405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:30.385405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:30.385405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:30.385405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:30.385405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:30.385405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:30.385405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:30.385405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:30.385405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:30.385405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:30.386898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:30.386898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:30.386898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:24:30.386898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:24:30.386898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:24:30.386898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:24:30.386898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:24:30.386898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:24:30.386898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:24:30.386898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:24:30.389979 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 622/1253] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:24:30.391025 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d349a-7897-460e-abef-b92f93e0752b None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:30.453717 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:30.501997 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d349a-7897-460e-abef-b92f93e0752b None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:30.540623 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7290e43-7578-4bf4-8550-afaf0169d975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:30.541213 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7290e43-7578-4bf4-8550-afaf0169d975 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:30.541484 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7290e43-7578-4bf4-8550-afaf0169d975 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:30.541764 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7290e43-7578-4bf4-8550-afaf0169d975 None admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:30.542570 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7290e43-7578-4bf4-8550-afaf0169d975 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:30.542874 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7290e43-7578-4bf4-8550-afaf0169d975 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:30.544308 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7290e43-7578-4bf4-8550-afaf0169d975 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:30.563820 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 632/1254] 10.222.0.18 () {56 vars in 1143 bytes} [Thu Sep 1 16:24:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:30.570539 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b88e0d-772c-4853-9d48-ec25a933051e None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:30.680857 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b88e0d-772c-4853-9d48-ec25a933051e None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:30.719587 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c77ad-ba7a-411f-ba6d-628261baa536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:30.720489 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c77ad-ba7a-411f-ba6d-628261baa536 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:30.720778 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c77ad-ba7a-411f-ba6d-628261baa536 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:30.721119 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c77ad-ba7a-411f-ba6d-628261baa536 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:30.780602 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:30.794463 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c77ad-ba7a-411f-ba6d-628261baa536 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:30.795903 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c77ad-ba7a-411f-ba6d-628261baa536 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:30.900285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 623/1255] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:24:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:24:30.908710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7290e43-7578-4bf4-8550-afaf0169d975 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:31.024453 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7290e43-7578-4bf4-8550-afaf0169d975 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:31.039645 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:31.060379 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dad25e-19dc-49bd-ac3f-1ed7ce3192ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:31.061005 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dad25e-19dc-49bd-ac3f-1ed7ce3192ca None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:31.061281 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dad25e-19dc-49bd-ac3f-1ed7ce3192ca None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:31.061549 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dad25e-19dc-49bd-ac3f-1ed7ce3192ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:31.071216 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dad25e-19dc-49bd-ac3f-1ed7ce3192ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:31.072653 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dad25e-19dc-49bd-ac3f-1ed7ce3192ca None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:31.088016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-f6dad25e-19dc-49bd-ac3f-1ed7ce3192ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:31.088016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:31.088016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:31.088016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:31.088016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:31.088016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:31.088016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:31.088016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:31.088016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:31.088016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:31.088016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:31.088016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:31.088016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:31.088016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:31.088016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:31.088016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:31.089179 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:31.089179 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:31.089179 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:31.089179 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:31.089179 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:31.089179 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:31.089179 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:31.089179 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:31.089179 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:31.089179 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:31.089179 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:31.089179 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:31.089179 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:31.089179 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:31.089179 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:31.089179 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:31.089179 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:31.090082 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:31.090082 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:31.090082 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:24:31.090082 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 633/1256] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:24:31.095163 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c77ad-ba7a-411f-ba6d-628261baa536 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:31.194147 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:31.208853 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c77ad-ba7a-411f-ba6d-628261baa536 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:31.246272 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556b67b-7952-4719-9ab8-79e2e7a1f46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:31.246894 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556b67b-7952-4719-9ab8-79e2e7a1f46a None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:31.247145 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556b67b-7952-4719-9ab8-79e2e7a1f46a None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:31.247410 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556b67b-7952-4719-9ab8-79e2e7a1f46a None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:31.248041 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e556b67b-7952-4719-9ab8-79e2e7a1f46a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:31.248340 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e556b67b-7952-4719-9ab8-79e2e7a1f46a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:31.249680 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e556b67b-7952-4719-9ab8-79e2e7a1f46a None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:31.266777 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 624/1257] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:31.267923 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dad25e-19dc-49bd-ac3f-1ed7ce3192ca None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:31.373856 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dad25e-19dc-49bd-ac3f-1ed7ce3192ca None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:31.410857 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbb234-68cc-4ab4-9d8d-d548997b98b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:31.411486 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbb234-68cc-4ab4-9d8d-d548997b98b2 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:31.411807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbb234-68cc-4ab4-9d8d-d548997b98b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:31.412134 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbb234-68cc-4ab4-9d8d-d548997b98b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:31.419295 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dbb234-68cc-4ab4-9d8d-d548997b98b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:31.420741 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dbb234-68cc-4ab4-9d8d-d548997b98b2 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:31.432895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-05dbb234-68cc-4ab4-9d8d-d548997b98b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:31.432895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:31.432895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:31.432895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:31.432895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:31.432895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:31.432895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:31.432895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:31.432895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:31.432895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:31.432895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:31.432895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:31.432895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:24:31.432895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:24:31.432895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:24:31.435246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:24:31.435246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:31.435246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:31.435246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:31.435246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:31.435246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:31.435246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:31.435246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:31.435246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:31.435246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:31.435246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:31.435246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:31.435246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:31.435246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:31.435246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:31.435246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:31.435246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:31.436588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:24:31.436588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:24:31.436588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:24:31.436588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:24:31.436588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:24:31.436588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:24:31.436588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:31.436588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:24:31.436588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 634/1258] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:24:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:31.440137 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556b67b-7952-4719-9ab8-79e2e7a1f46a None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:31.550752 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556b67b-7952-4719-9ab8-79e2e7a1f46a None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:31.585945 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b96b6c-d957-4f4d-8f82-14593291e96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:31.586638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b96b6c-d957-4f4d-8f82-14593291e96c None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:31.586869 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b96b6c-d957-4f4d-8f82-14593291e96c None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:31.587148 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b96b6c-d957-4f4d-8f82-14593291e96c None admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:31.587839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b96b6c-d957-4f4d-8f82-14593291e96c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:31.588171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b96b6c-d957-4f4d-8f82-14593291e96c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:31.589583 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b96b6c-d957-4f4d-8f82-14593291e96c None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:31.601811 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 625/1259] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:24:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:31.608124 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbb234-68cc-4ab4-9d8d-d548997b98b2 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:31.638011 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:31.719497 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbb234-68cc-4ab4-9d8d-d548997b98b2 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:31.756316 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762ea09-3f16-40fc-9254-9698297b5e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:31.756929 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762ea09-3f16-40fc-9254-9698297b5e98 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:31.757223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762ea09-3f16-40fc-9254-9698297b5e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:31.757482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762ea09-3f16-40fc-9254-9698297b5e98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:31.758261 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9762ea09-3f16-40fc-9254-9698297b5e98 None admin] RBAC: Adding query filter params (role.id=e8ba063905484d22993bcf04830091a3, scope.project.id=36ce80c4e0d64562a9ab3961be76fad9, user.id=3ddfb151bbf844efae73ec1990e7eaa3) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:31.758583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9762ea09-3f16-40fc-9254-9698297b5e98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:31.760124 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9762ea09-3f16-40fc-9254-9698297b5e98 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:31.766405 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 635/1260] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:24:31 2022] GET /identity/v3/role_assignments?user.id=3ddfb151bbf844efae73ec1990e7eaa3&scope.project.id=36ce80c4e0d64562a9ab3961be76fad9&role.id=e8ba063905484d22993bcf04830091a3 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:33.647235 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757a665-2a4b-45f0-aed9-039ab963c17a None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:33.647735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757a665-2a4b-45f0-aed9-039ab963c17a None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:33.647735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757a665-2a4b-45f0-aed9-039ab963c17a None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:33.649035 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 626/1261] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:33.654418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31d596-985c-48a0-81f5-4ce19071af34 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:33.654809 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31d596-985c-48a0-81f5-4ce19071af34 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:33.654979 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31d596-985c-48a0-81f5-4ce19071af34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:33.673755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-4d31d596-985c-48a0-81f5-4ce19071af34 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:33.674953 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-4d31d596-985c-48a0-81f5-4ce19071af34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:33.739887 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 636/1262] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:34.004984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6757a665-2a4b-45f0-aed9-039ab963c17a None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:34.015843 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:34.103589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6757a665-2a4b-45f0-aed9-039ab963c17a None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:34.136214 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6727aa-31f5-452d-898e-ab08ad7d06ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:34.136828 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6727aa-31f5-452d-898e-ab08ad7d06ed None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:34.137106 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6727aa-31f5-452d-898e-ab08ad7d06ed None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:34.137375 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6727aa-31f5-452d-898e-ab08ad7d06ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:34.147640 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6727aa-31f5-452d-898e-ab08ad7d06ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:24:34.152471 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-dd6727aa-31f5-452d-898e-ab08ad7d06ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:24:34.152471 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:34.152471 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:34.152471 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:34.152471 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:34.152471 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:34.152471 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:34.152471 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:34.152471 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:34.152471 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:34.152471 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:34.152471 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:34.152471 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:24:34.152471 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:24:34.152471 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:24:34.152471 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:24:34.153574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:24:34.153574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:24:34.153574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:24:34.153574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:24:34.153574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:34.153574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:34.153574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:34.153574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:34.153574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:34.153574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:34.153574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:34.153574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:34.153574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:34.153574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:34.153574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:34.153574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:34.154797 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:34.154797 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:34.154797 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:34.154797 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:34.154797 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:24:34.154797 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:24:34.154797 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:24:34.154797 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:24:34.154797 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:24:34.154797 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:24:34.154797 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:24:34.154797 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:24:34.154797 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 627/1263] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:24:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:34.159608 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d31d596-985c-48a0-81f5-4ce19071af34 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:34.177236 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:34.255371 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d31d596-985c-48a0-81f5-4ce19071af34 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:34.288087 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0e4e8-963b-4d70-86a9-5f82fbf35822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:34.288687 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0e4e8-963b-4d70-86a9-5f82fbf35822 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:34.288970 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0e4e8-963b-4d70-86a9-5f82fbf35822 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:34.289243 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0e4e8-963b-4d70-86a9-5f82fbf35822 None admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:34.289899 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf0e4e8-963b-4d70-86a9-5f82fbf35822 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:34.290203 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf0e4e8-963b-4d70-86a9-5f82fbf35822 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:34.291600 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf0e4e8-963b-4d70-86a9-5f82fbf35822 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:34.299366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 637/1264] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:24:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:34.305017 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6727aa-31f5-452d-898e-ab08ad7d06ed None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:34.371347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:34.405008 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6727aa-31f5-452d-898e-ab08ad7d06ed None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:34.437074 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed69e1a-3afc-4f88-b2cd-ff67aae030ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:34.437692 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed69e1a-3afc-4f88-b2cd-ff67aae030ad None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:34.437971 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed69e1a-3afc-4f88-b2cd-ff67aae030ad None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:34.438268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed69e1a-3afc-4f88-b2cd-ff67aae030ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:34.448417 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed69e1a-3afc-4f88-b2cd-ff67aae030ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:34.449827 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed69e1a-3afc-4f88-b2cd-ff67aae030ad None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:34.464439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-5ed69e1a-3afc-4f88-b2cd-ff67aae030ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:34.464439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:34.464439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:34.464439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:34.464439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:34.464439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:34.464439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:34.464439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:34.464439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:34.464439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:34.464439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:34.464439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:34.464439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:34.464439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:34.464439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:34.464439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:34.465534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:34.465534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:34.465534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:34.465534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:34.465534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:34.465534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:34.465534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:34.465534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:34.465534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:34.465534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:34.465534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:34.465534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:34.465534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:34.465534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:34.465534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:34.465534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:34.465534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:34.466443 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:34.466443 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:34.466443 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:24:34.466443 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 628/1265] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:34.471240 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0e4e8-963b-4d70-86a9-5f82fbf35822 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:34.549130 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:34.568980 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0e4e8-963b-4d70-86a9-5f82fbf35822 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:34.600327 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d74a7-5172-49ad-8b08-8ef12ada0d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:34.600920 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d74a7-5172-49ad-8b08-8ef12ada0d4f None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:34.601198 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d74a7-5172-49ad-8b08-8ef12ada0d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:34.601470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d74a7-5172-49ad-8b08-8ef12ada0d4f None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:34.602097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d74a7-5172-49ad-8b08-8ef12ada0d4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:34.602433 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d74a7-5172-49ad-8b08-8ef12ada0d4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:34.603786 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d74a7-5172-49ad-8b08-8ef12ada0d4f None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:34.615714 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 638/1266] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:34.621190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed69e1a-3afc-4f88-b2cd-ff67aae030ad None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:34.718008 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed69e1a-3afc-4f88-b2cd-ff67aae030ad None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:34.736826 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:34.752093 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5653d3-8b5c-4d79-b2c6-59cb09b55aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:34.752705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5653d3-8b5c-4d79-b2c6-59cb09b55aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:34.752982 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5653d3-8b5c-4d79-b2c6-59cb09b55aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:34.753788 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5653d3-8b5c-4d79-b2c6-59cb09b55aa1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:34.760994 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5653d3-8b5c-4d79-b2c6-59cb09b55aa1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:34.762434 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5653d3-8b5c-4d79-b2c6-59cb09b55aa1 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:34.773184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-7b5653d3-8b5c-4d79-b2c6-59cb09b55aa1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:24:34.773184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:34.773184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:34.773184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:34.773184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:34.773184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:34.773184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:34.773184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:34.773184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:34.773184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:34.773184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:34.773184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:34.773184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:24:34.773184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:24:34.773184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:24:34.773184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:24:34.774285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:34.774285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:34.774285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:24:34.774285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:34.774285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:24:34.774285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:34.774285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:34.774285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:34.774285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:34.774285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:34.774285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:34.774285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:34.774285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:34.774285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:34.774285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:34.774285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:34.774285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:34.774285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:34.775472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:34.775472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:34.775472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:24:34.775472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:24:34.775472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:24:34.775472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:24:34.775472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:24:34.775472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:24:34.775472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:24:34.775472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:24:34.775472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 629/1267] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:34.780073 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d74a7-5172-49ad-8b08-8ef12ada0d4f None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:34.877535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d74a7-5172-49ad-8b08-8ef12ada0d4f None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:34.921780 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750abc7-bb40-4a83-8c1c-08f8c7453b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:34.922592 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750abc7-bb40-4a83-8c1c-08f8c7453b72 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:34.922896 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750abc7-bb40-4a83-8c1c-08f8c7453b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:34.923203 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750abc7-bb40-4a83-8c1c-08f8c7453b72 None admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:34.924035 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0750abc7-bb40-4a83-8c1c-08f8c7453b72 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:34.924436 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0750abc7-bb40-4a83-8c1c-08f8c7453b72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:34.926069 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0750abc7-bb40-4a83-8c1c-08f8c7453b72 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:34.935329 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:34.944329 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 639/1268] 10.222.0.18 () {56 vars in 1143 bytes} [Thu Sep 1 16:24:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:34.950051 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5653d3-8b5c-4d79-b2c6-59cb09b55aa1 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:35.048222 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5653d3-8b5c-4d79-b2c6-59cb09b55aa1 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:35.080667 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067c137-d5cf-4937-8e3e-abaa220f4391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:35.081310 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067c137-d5cf-4937-8e3e-abaa220f4391 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:35.081595 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067c137-d5cf-4937-8e3e-abaa220f4391 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:35.081868 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067c137-d5cf-4937-8e3e-abaa220f4391 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:35.092045 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7067c137-d5cf-4937-8e3e-abaa220f4391 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:35.093472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7067c137-d5cf-4937-8e3e-abaa220f4391 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:35.107864 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-7067c137-d5cf-4937-8e3e-abaa220f4391 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:35.107864 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:35.107864 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:35.107864 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:35.107864 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:35.107864 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:35.107864 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:35.107864 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:35.107864 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:35.107864 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:35.107864 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:35.107864 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:35.107864 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:35.107864 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:35.107864 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:35.107864 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:35.108919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:35.108919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:35.108919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:35.108919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:35.108919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:35.108919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:35.108919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:35.108919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:35.108919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:35.108919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:35.108919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:35.108919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:35.108919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:35.108919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:35.108919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:35.108919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:35.108919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:35.110179 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:35.110179 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:35.110179 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:24:35.110179 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 630/1269] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:35.115610 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750abc7-bb40-4a83-8c1c-08f8c7453b72 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:35.210263 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750abc7-bb40-4a83-8c1c-08f8c7453b72 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:35.244441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b006673-5773-4bc7-a842-e471dedd1e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:35.245036 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b006673-5773-4bc7-a842-e471dedd1e7d None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:35.245316 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b006673-5773-4bc7-a842-e471dedd1e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:35.245586 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b006673-5773-4bc7-a842-e471dedd1e7d None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:35.246211 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b006673-5773-4bc7-a842-e471dedd1e7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:35.246545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b006673-5773-4bc7-a842-e471dedd1e7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:35.247900 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b006673-5773-4bc7-a842-e471dedd1e7d None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:35.259956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 640/1270] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:35.265558 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067c137-d5cf-4937-8e3e-abaa220f4391 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:35.297463 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:35.368739 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067c137-d5cf-4937-8e3e-abaa220f4391 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:35.401300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2741d71-b67d-4e25-91cf-afe3fadf80da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:35.401928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2741d71-b67d-4e25-91cf-afe3fadf80da None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:35.402207 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2741d71-b67d-4e25-91cf-afe3fadf80da None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:35.402512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2741d71-b67d-4e25-91cf-afe3fadf80da None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:35.409008 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2741d71-b67d-4e25-91cf-afe3fadf80da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:35.411107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2741d71-b67d-4e25-91cf-afe3fadf80da None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:35.422060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-c2741d71-b67d-4e25-91cf-afe3fadf80da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:35.422060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:35.422060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:35.422060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:35.422060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:35.422060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:35.422060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:35.422060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:35.422060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:35.422060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:35.422060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:35.422060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:35.422060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:24:35.422060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:24:35.422060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:24:35.423088 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:24:35.423088 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:35.423088 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:35.423088 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:35.423088 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:35.423088 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:35.423088 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:35.423088 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:35.423088 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:35.423088 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:35.423088 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:35.423088 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:35.423088 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:35.423088 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:35.423088 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:35.423088 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:35.423088 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:35.424325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:24:35.424325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:24:35.424325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:24:35.424325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:24:35.424325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:24:35.424325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:24:35.424325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:35.424325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:24:35.424325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 631/1271] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:24:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:35.429447 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b006673-5773-4bc7-a842-e471dedd1e7d None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:35.489248 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:35.531032 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b006673-5773-4bc7-a842-e471dedd1e7d None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:35.566752 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd4fa39-d98c-4524-99be-829ab4f04443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:35.567351 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4fa39-d98c-4524-99be-829ab4f04443 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:35.567630 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4fa39-d98c-4524-99be-829ab4f04443 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:35.567902 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4fa39-d98c-4524-99be-829ab4f04443 None admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:35.568585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd4fa39-d98c-4524-99be-829ab4f04443 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:35.568892 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd4fa39-d98c-4524-99be-829ab4f04443 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:35.570331 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd4fa39-d98c-4524-99be-829ab4f04443 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:35.582293 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 641/1272] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:24:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:35.588157 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2741d71-b67d-4e25-91cf-afe3fadf80da None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:35.676259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:35.685184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2741d71-b67d-4e25-91cf-afe3fadf80da None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:35.718684 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d65ca-db77-4126-9bdc-784c62e9e1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:35.719356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d65ca-db77-4126-9bdc-784c62e9e1e8 None admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:35.719638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d65ca-db77-4126-9bdc-784c62e9e1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:35.719911 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d65ca-db77-4126-9bdc-784c62e9e1e8 None admin] PATH_INFO: `/v3/projects/36ce80c4e0d64562a9ab3961be76fad9/users/3ddfb151bbf844efae73ec1990e7eaa3/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:35.721790 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d65ca-db77-4126-9bdc-784c62e9e1e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=36ce80c4e0d64562a9ab3961be76fad9, user_id=3ddfb151bbf844efae73ec1990e7eaa3, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:35.723236 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d65ca-db77-4126-9bdc-784c62e9e1e8 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:35.735212 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 632/1273] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Sep 1 16:24:35 2022] PUT /identity/v3/projects/36ce80c4e0d64562a9ab3961be76fad9/users/3ddfb151bbf844efae73ec1990e7eaa3/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:24:37.603209 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d4a32-c029-4ae0-a7d5-f8728a6aa8b8 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:37.603755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d4a32-c029-4ae0-a7d5-f8728a6aa8b8 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:37.603755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d4a32-c029-4ae0-a7d5-f8728a6aa8b8 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:37.605110 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 642/1274] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:37.610072 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986c3a2-5384-431f-8022-db811fb57aaf None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:37.610460 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986c3a2-5384-431f-8022-db811fb57aaf None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:37.610677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986c3a2-5384-431f-8022-db811fb57aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:37.628843 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-3986c3a2-5384-431f-8022-db811fb57aaf None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:37.629961 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3986c3a2-5384-431f-8022-db811fb57aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:37.693931 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 633/1275] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:37.966360 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d4a32-c029-4ae0-a7d5-f8728a6aa8b8 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:38.068367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d4a32-c029-4ae0-a7d5-f8728a6aa8b8 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:38.086136 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:38.104002 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c9aa10-b89d-49d8-83b9-54b14fe6f8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:38.104627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9aa10-b89d-49d8-83b9-54b14fe6f8af None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:38.104912 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9aa10-b89d-49d8-83b9-54b14fe6f8af None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:38.105189 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9aa10-b89d-49d8-83b9-54b14fe6f8af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:38.116585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c9aa10-b89d-49d8-83b9-54b14fe6f8af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:24:38.124578 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-c0c9aa10-b89d-49d8-83b9-54b14fe6f8af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:24:38.124578 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:38.124578 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:38.124578 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:38.124578 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:38.124578 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:38.124578 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:38.124578 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:38.124578 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:38.124578 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:38.124578 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:38.124578 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:38.124578 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:24:38.124578 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:24:38.124578 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:24:38.124578 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:24:38.125664 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:24:38.125664 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:24:38.125664 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:24:38.125664 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:24:38.125664 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:38.125664 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:38.125664 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:38.125664 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:38.125664 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:38.125664 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:38.125664 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:38.125664 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:38.125664 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:38.125664 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:38.125664 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:38.125664 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:38.126675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:38.126675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:38.126675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:38.126675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:38.126675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:24:38.126675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:24:38.126675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:24:38.126675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:24:38.126675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:24:38.126675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:24:38.126675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:24:38.126675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:24:38.126675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 643/1276] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Sep 1 16:24:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:38.132585 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986c3a2-5384-431f-8022-db811fb57aaf None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:38.211712 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:38.230161 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986c3a2-5384-431f-8022-db811fb57aaf None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:38.263533 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d565c-38fc-43e8-abb4-103ec9ab3510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:38.264127 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d565c-38fc-43e8-abb4-103ec9ab3510 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:38.264405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d565c-38fc-43e8-abb4-103ec9ab3510 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:38.264674 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d565c-38fc-43e8-abb4-103ec9ab3510 None admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:38.265323 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d565c-38fc-43e8-abb4-103ec9ab3510 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:38.265623 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d565c-38fc-43e8-abb4-103ec9ab3510 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:38.267014 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d565c-38fc-43e8-abb4-103ec9ab3510 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:38.274801 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 634/1277] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Sep 1 16:24:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:38.280923 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c9aa10-b89d-49d8-83b9-54b14fe6f8af None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:38.382387 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c9aa10-b89d-49d8-83b9-54b14fe6f8af None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:38.415352 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9519a8a-5669-457e-b616-1cbbe9eecccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:38.415965 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9519a8a-5669-457e-b616-1cbbe9eecccf None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:38.416242 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9519a8a-5669-457e-b616-1cbbe9eecccf None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:38.416547 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9519a8a-5669-457e-b616-1cbbe9eecccf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:38.426595 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9519a8a-5669-457e-b616-1cbbe9eecccf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:38.428021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9519a8a-5669-457e-b616-1cbbe9eecccf None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:38.443050 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-d9519a8a-5669-457e-b616-1cbbe9eecccf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:38.443050 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:38.443050 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:38.443050 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:38.443050 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:38.443050 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:38.443050 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:38.443050 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:38.443050 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:38.443050 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:38.443050 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:38.443050 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:38.443050 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:38.443050 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:38.443050 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:38.443050 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:38.444229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:38.444229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:38.444229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:38.444229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:38.444229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:38.444229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:38.444229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:38.444229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:38.444229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:38.444229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:38.444229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:38.444229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:38.444229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:38.444229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:38.444229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:38.444229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:38.444229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:38.445369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:38.445369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:38.445369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:24:38.445369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 644/1278] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:38.450169 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d565c-38fc-43e8-abb4-103ec9ab3510 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:38.546592 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d565c-38fc-43e8-abb4-103ec9ab3510 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:38.579466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff2e35-4d97-44ff-b170-5f1f406cd8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:38.580059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff2e35-4d97-44ff-b170-5f1f406cd8f3 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:38.580342 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff2e35-4d97-44ff-b170-5f1f406cd8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:38.580614 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff2e35-4d97-44ff-b170-5f1f406cd8f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:38.581273 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adff2e35-4d97-44ff-b170-5f1f406cd8f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:38.581580 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adff2e35-4d97-44ff-b170-5f1f406cd8f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:38.582953 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adff2e35-4d97-44ff-b170-5f1f406cd8f3 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:38.594221 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 635/1279] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:38.599718 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9519a8a-5669-457e-b616-1cbbe9eecccf None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:38.632072 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:38.700201 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9519a8a-5669-457e-b616-1cbbe9eecccf None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:38.734211 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d1897-eb0d-46f1-9c2e-1d317eb544c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:38.734845 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d1897-eb0d-46f1-9c2e-1d317eb544c4 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:38.735125 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d1897-eb0d-46f1-9c2e-1d317eb544c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:38.735398 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d1897-eb0d-46f1-9c2e-1d317eb544c4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:38.745364 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d1897-eb0d-46f1-9c2e-1d317eb544c4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:38.747304 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d1897-eb0d-46f1-9c2e-1d317eb544c4 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:38.758299 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-991d1897-eb0d-46f1-9c2e-1d317eb544c4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:24:38.758299 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:38.758299 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:38.758299 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:38.758299 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:38.758299 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:38.758299 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:38.758299 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:38.758299 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:38.758299 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:38.758299 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:38.758299 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:38.758299 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:24:38.758299 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:24:38.758299 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:24:38.758299 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:24:38.759341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:38.759341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:38.759341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:24:38.759341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:38.759341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:24:38.759341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:38.759341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:38.759341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:38.759341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:38.759341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:38.759341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:38.759341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:38.759341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:38.759341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:38.759341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:38.759341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:38.759341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:38.759341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:38.760306 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:38.760306 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:38.760306 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:24:38.760306 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:24:38.760306 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:24:38.760306 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:24:38.760306 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:24:38.760306 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:24:38.760306 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:24:38.760306 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:24:38.760306 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 645/1280] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:38.765008 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff2e35-4d97-44ff-b170-5f1f406cd8f3 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:38.768787 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:38.862094 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff2e35-4d97-44ff-b170-5f1f406cd8f3 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:38.904045 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36b4e5-fda2-4f2d-8382-1f1dfb1a3253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:38.904687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36b4e5-fda2-4f2d-8382-1f1dfb1a3253 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:38.905128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36b4e5-fda2-4f2d-8382-1f1dfb1a3253 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:38.905503 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36b4e5-fda2-4f2d-8382-1f1dfb1a3253 None admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:38.906333 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36b4e5-fda2-4f2d-8382-1f1dfb1a3253 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:38.906730 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36b4e5-fda2-4f2d-8382-1f1dfb1a3253 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:38.908271 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36b4e5-fda2-4f2d-8382-1f1dfb1a3253 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:38.929171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 636/1281] 10.222.0.18 () {56 vars in 1143 bytes} [Thu Sep 1 16:24:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:38.935031 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d1897-eb0d-46f1-9c2e-1d317eb544c4 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:39.023906 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:39.033492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d1897-eb0d-46f1-9c2e-1d317eb544c4 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:39.065697 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da2f3f-3bfa-4430-b208-aecf4ec7159a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:39.066317 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da2f3f-3bfa-4430-b208-aecf4ec7159a None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:39.066601 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da2f3f-3bfa-4430-b208-aecf4ec7159a None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:39.066874 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da2f3f-3bfa-4430-b208-aecf4ec7159a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:39.131071 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da2f3f-3bfa-4430-b208-aecf4ec7159a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:39.132454 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da2f3f-3bfa-4430-b208-aecf4ec7159a None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:39.216242 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:39.226992 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 646/1282] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Sep 1 16:24:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:24:39.232717 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36b4e5-fda2-4f2d-8382-1f1dfb1a3253 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:39.292225 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:39.328887 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36b4e5-fda2-4f2d-8382-1f1dfb1a3253 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:39.361909 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170077b-6e6a-49d2-84a9-931f9b1dac62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:39.362597 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170077b-6e6a-49d2-84a9-931f9b1dac62 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:39.362882 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170077b-6e6a-49d2-84a9-931f9b1dac62 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:39.363155 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170077b-6e6a-49d2-84a9-931f9b1dac62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:39.373166 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6170077b-6e6a-49d2-84a9-931f9b1dac62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:39.374609 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6170077b-6e6a-49d2-84a9-931f9b1dac62 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:39.389162 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-6170077b-6e6a-49d2-84a9-931f9b1dac62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:39.389162 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:39.389162 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:39.389162 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:39.389162 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:39.389162 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:39.389162 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:39.389162 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:39.389162 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:39.389162 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:39.389162 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:39.389162 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:39.389162 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:39.389162 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:39.389162 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:39.389162 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:39.390275 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:39.390275 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:39.390275 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:39.390275 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:39.390275 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:39.390275 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:39.390275 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:39.390275 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:39.390275 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:39.390275 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:39.390275 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:39.390275 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:39.390275 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:39.390275 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:39.390275 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:39.390275 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:39.390275 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:39.391605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:39.391605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:39.391605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:24:39.391605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 637/1283] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:39.396489 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da2f3f-3bfa-4430-b208-aecf4ec7159a None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:39.494587 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da2f3f-3bfa-4430-b208-aecf4ec7159a None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:39.528923 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c1b6bd-f3c9-43a7-b81d-a5f826211b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:39.529515 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1b6bd-f3c9-43a7-b81d-a5f826211b15 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:39.529799 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1b6bd-f3c9-43a7-b81d-a5f826211b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:39.530075 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1b6bd-f3c9-43a7-b81d-a5f826211b15 None admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:39.530736 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c1b6bd-f3c9-43a7-b81d-a5f826211b15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:39.531042 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c1b6bd-f3c9-43a7-b81d-a5f826211b15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:39.532396 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c1b6bd-f3c9-43a7-b81d-a5f826211b15 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:39.543243 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 647/1284] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:39.548879 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170077b-6e6a-49d2-84a9-931f9b1dac62 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:39.646489 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170077b-6e6a-49d2-84a9-931f9b1dac62 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:39.659652 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:39.679029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c3035-7512-4345-b386-a2a1eea4f361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:39.679644 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c3035-7512-4345-b386-a2a1eea4f361 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:39.679923 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c3035-7512-4345-b386-a2a1eea4f361 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:39.680196 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c3035-7512-4345-b386-a2a1eea4f361 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:39.686614 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844c3035-7512-4345-b386-a2a1eea4f361 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:39.688037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844c3035-7512-4345-b386-a2a1eea4f361 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:39.699424 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application [None req-844c3035-7512-4345-b386-a2a1eea4f361 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:39.699424 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:39.699424 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:39.699424 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:39.699424 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:39.699424 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:39.699424 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:39.699424 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:39.699424 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:39.699424 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:39.699424 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:39.699424 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:39.699424 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:24:39.699424 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:24:39.699424 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:24:39.700405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:24:39.700405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:39.700405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:39.700405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:39.700405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:39.700405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:39.700405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:39.700405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:39.700405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:39.700405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:39.700405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:39.700405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:39.700405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:39.700405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:39.700405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:39.700405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:39.700405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:39.701481 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:24:39.701481 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:24:39.701481 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:24:39.701481 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:24:39.701481 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:24:39.701481 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:24:39.701481 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:39.701481 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: ERROR keystone.server.flask.application Sep 01 16:24:39.701481 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 638/1285] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Sep 1 16:24:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:39.707049 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c1b6bd-f3c9-43a7-b81d-a5f826211b15 None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:39.757975 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:39.807749 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c1b6bd-f3c9-43a7-b81d-a5f826211b15 None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:39.841346 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c5c87-b471-473a-97b8-c69cb85c8c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:39.841944 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c5c87-b471-473a-97b8-c69cb85c8c7f None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:39.842229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c5c87-b471-473a-97b8-c69cb85c8c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:39.842535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c5c87-b471-473a-97b8-c69cb85c8c7f None admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:39.843222 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c5c87-b471-473a-97b8-c69cb85c8c7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:39.843535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c5c87-b471-473a-97b8-c69cb85c8c7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:39.844960 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c5c87-b471-473a-97b8-c69cb85c8c7f None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:39.858197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 648/1286] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Sep 1 16:24:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:39.864885 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c3035-7512-4345-b386-a2a1eea4f361 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:39.971168 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c3035-7512-4345-b386-a2a1eea4f361 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:40.005489 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5086c-2621-428c-88e1-0dd9d786ba95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:40.006084 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5086c-2621-428c-88e1-0dd9d786ba95 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:40.006390 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5086c-2621-428c-88e1-0dd9d786ba95 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:40.006659 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5086c-2621-428c-88e1-0dd9d786ba95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:40.007370 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d5086c-2621-428c-88e1-0dd9d786ba95 None admin] RBAC: Adding query filter params (role.id=e8ba063905484d22993bcf04830091a3, scope.project.id=36ce80c4e0d64562a9ab3961be76fad9, user.id=3ddfb151bbf844efae73ec1990e7eaa3) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:40.007672 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d5086c-2621-428c-88e1-0dd9d786ba95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:40.009085 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d5086c-2621-428c-88e1-0dd9d786ba95 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:40.014661 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 639/1287] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Sep 1 16:24:39 2022] GET /identity/v3/role_assignments?user.id=3ddfb151bbf844efae73ec1990e7eaa3&scope.project.id=36ce80c4e0d64562a9ab3961be76fad9&role.id=e8ba063905484d22993bcf04830091a3 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:41.915639 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daefe89a-530d-4fec-ac2e-8dbf5b952b9e None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:41.916225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daefe89a-530d-4fec-ac2e-8dbf5b952b9e None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:41.916509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daefe89a-530d-4fec-ac2e-8dbf5b952b9e None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:41.918423 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 649/1288] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:24:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:41.923402 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86aa7d-e0fc-4f43-b23c-87618286a108 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:41.923806 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86aa7d-e0fc-4f43-b23c-87618286a108 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:41.923956 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86aa7d-e0fc-4f43-b23c-87618286a108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:41.942637 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-ad86aa7d-e0fc-4f43-b23c-87618286a108 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:41.943815 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ad86aa7d-e0fc-4f43-b23c-87618286a108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:41.972958 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:42.009774 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 640/1289] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:42.276116 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daefe89a-530d-4fec-ac2e-8dbf5b952b9e None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:42.396000 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daefe89a-530d-4fec-ac2e-8dbf5b952b9e None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:42.408670 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:42.435797 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c64e15-9f93-480a-bf44-1f5c8870154f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:42.436610 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c64e15-9f93-480a-bf44-1f5c8870154f None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:42.437049 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c64e15-9f93-480a-bf44-1f5c8870154f None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:42.437480 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c64e15-9f93-480a-bf44-1f5c8870154f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:42.438584 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c64e15-9f93-480a-bf44-1f5c8870154f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:42.440877 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c64e15-9f93-480a-bf44-1f5c8870154f None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:42.454014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-75c64e15-9f93-480a-bf44-1f5c8870154f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:24:42.454014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:42.454014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:24:42.454014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:24:42.454014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:24:42.454014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:24:42.454014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:24:42.454014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:42.454014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:24:42.454014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:24:42.454014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:24:42.454014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:24:42.454014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:42.454014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:42.454014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:42.454014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:42.454014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:42.455286 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:42.455286 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:42.455286 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:42.455286 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:42.455286 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:42.455286 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:42.455286 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:24:42.455286 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:24:42.455286 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:24:42.455286 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:24:42.455286 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:42.455286 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:42.455286 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:42.455286 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:42.455286 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:42.455286 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:42.457230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:42.457230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:42.457230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:42.457230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:42.457230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:42.457230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:42.457230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:42.457230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:42.457230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:42.457230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:42.457230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:24:42.457230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:42.457230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:24:42.457230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:24:42.457230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 650/1290] 10.222.0.18 () {56 vars in 1117 bytes} [Thu Sep 1 16:24:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:42.462356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86aa7d-e0fc-4f43-b23c-87618286a108 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:42.555456 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86aa7d-e0fc-4f43-b23c-87618286a108 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:42.588267 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2f68a-039b-4669-877c-9a1fee02f06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:42.588856 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2f68a-039b-4669-877c-9a1fee02f06b None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:42.589132 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2f68a-039b-4669-877c-9a1fee02f06b None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:42.589403 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2f68a-039b-4669-877c-9a1fee02f06b None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:42.590017 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e2f68a-039b-4669-877c-9a1fee02f06b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:42.590341 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e2f68a-039b-4669-877c-9a1fee02f06b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:42.591676 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e2f68a-039b-4669-877c-9a1fee02f06b None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:42.596286 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 641/1291] 10.222.0.18 () {56 vars in 1116 bytes} [Thu Sep 1 16:24:42 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:42.601887 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c64e15-9f93-480a-bf44-1f5c8870154f None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:42.699202 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c64e15-9f93-480a-bf44-1f5c8870154f None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:42.731183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93d709-b17d-4cc9-8624-fda99b0f4448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:42.731774 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93d709-b17d-4cc9-8624-fda99b0f4448 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:42.732055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93d709-b17d-4cc9-8624-fda99b0f4448 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:42.732330 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93d709-b17d-4cc9-8624-fda99b0f4448 None admin] PATH_INFO: `/v3/services` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:42.732946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd93d709-b17d-4cc9-8624-fda99b0f4448 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:42.733248 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd93d709-b17d-4cc9-8624-fda99b0f4448 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:42.734645 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd93d709-b17d-4cc9-8624-fda99b0f4448 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:42.739665 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 651/1292] 10.222.0.18 () {56 vars in 1116 bytes} [Thu Sep 1 16:24:42 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:44.616982 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790b45e-e948-4941-b0a9-6ba69b87e2a4 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:44.617573 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790b45e-e948-4941-b0a9-6ba69b87e2a4 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:44.617573 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790b45e-e948-4941-b0a9-6ba69b87e2a4 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:44.618914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 642/1293] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:24:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:44.623979 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3c521-7c90-49b1-bda6-ad82f1874fb0 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:44.624364 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3c521-7c90-49b1-bda6-ad82f1874fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:44.624556 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3c521-7c90-49b1-bda6-ad82f1874fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:44.642717 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-55e3c521-7c90-49b1-bda6-ad82f1874fb0 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:44.643884 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-55e3c521-7c90-49b1-bda6-ad82f1874fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:44.672251 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:44.708300 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 652/1294] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:44.981295 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790b45e-e948-4941-b0a9-6ba69b87e2a4 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:45.012250 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:45.079569 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790b45e-e948-4941-b0a9-6ba69b87e2a4 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:45.112426 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae2f64c-5eeb-4599-9150-c75afc53a481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:45.113026 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2f64c-5eeb-4599-9150-c75afc53a481 None admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:45.113568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2f64c-5eeb-4599-9150-c75afc53a481 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:45.113840 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2f64c-5eeb-4599-9150-c75afc53a481 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:45.114637 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae2f64c-5eeb-4599-9150-c75afc53a481 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:45.115989 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae2f64c-5eeb-4599-9150-c75afc53a481 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:45.129370 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 643/1295] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Sep 1 16:24:44 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:24:47.020460 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb0c45-77b4-495b-bc54-f5b5cbba73ea None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:47.021016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb0c45-77b4-495b-bc54-f5b5cbba73ea None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:47.021016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb0c45-77b4-495b-bc54-f5b5cbba73ea None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:47.022383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 653/1296] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:24:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:47.027369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370010e4-edec-4525-946d-9b6942395b33 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:47.027750 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370010e4-edec-4525-946d-9b6942395b33 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:47.027934 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370010e4-edec-4525-946d-9b6942395b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:47.046363 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-370010e4-edec-4525-946d-9b6942395b33 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:47.047564 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-370010e4-edec-4525-946d-9b6942395b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:47.118916 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 644/1297] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:47 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:47.386863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb0c45-77b4-495b-bc54-f5b5cbba73ea None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:47.483659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb0c45-77b4-495b-bc54-f5b5cbba73ea None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:47.516902 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139cc2e-45d0-4d21-8201-b92a0fed8d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:47.517518 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139cc2e-45d0-4d21-8201-b92a0fed8d7a None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:47.517799 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139cc2e-45d0-4d21-8201-b92a0fed8d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:47.518105 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139cc2e-45d0-4d21-8201-b92a0fed8d7a None admin] PATH_INFO: `/v3/services/placement` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:47.518720 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4139cc2e-45d0-4d21-8201-b92a0fed8d7a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:47.520078 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4139cc2e-45d0-4d21-8201-b92a0fed8d7a None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:47.528545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-4139cc2e-45d0-4d21-8201-b92a0fed8d7a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:24:47.528545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:47.528545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:24:47.528545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:24:47.528545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:24:47.528545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:24:47.528545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:24:47.528545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:47.528545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:24:47.528545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:24:47.528545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:24:47.528545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:24:47.528545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:47.528545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:47.528545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:47.528545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:47.528545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:47.529685 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:47.529685 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:47.529685 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:47.529685 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:47.529685 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:47.529685 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:47.529685 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:24:47.529685 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:24:47.529685 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:24:47.529685 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:24:47.529685 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:47.529685 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:47.529685 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:47.529685 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:47.529685 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:47.529685 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:47.530924 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:47.530924 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:47.530924 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:47.530924 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:47.530924 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:47.530924 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:47.530924 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:47.530924 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:47.530924 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:47.530924 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:47.530924 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:24:47.530924 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:47.530924 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:24:47.530924 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:24:47.530924 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 654/1298] 10.222.0.18 () {56 vars in 1117 bytes} [Thu Sep 1 16:24:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:47.535472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370010e4-edec-4525-946d-9b6942395b33 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:47.556082 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:47.636141 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370010e4-edec-4525-946d-9b6942395b33 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:47.668538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29366d2-d507-42dd-8251-81f29ac220e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:47.669136 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29366d2-d507-42dd-8251-81f29ac220e5 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:47.669412 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29366d2-d507-42dd-8251-81f29ac220e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:47.669682 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29366d2-d507-42dd-8251-81f29ac220e5 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:47.670328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29366d2-d507-42dd-8251-81f29ac220e5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:47.670651 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29366d2-d507-42dd-8251-81f29ac220e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:47.671978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29366d2-d507-42dd-8251-81f29ac220e5 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:47.677518 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 645/1299] 10.222.0.18 () {56 vars in 1116 bytes} [Thu Sep 1 16:24:47 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:47.683495 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139cc2e-45d0-4d21-8201-b92a0fed8d7a None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:47.713789 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:47.780738 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139cc2e-45d0-4d21-8201-b92a0fed8d7a None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:47.813807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c6cb9-f80f-4370-913b-9abbfab54e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:47.814424 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c6cb9-f80f-4370-913b-9abbfab54e18 None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:47.814705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c6cb9-f80f-4370-913b-9abbfab54e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:47.815000 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c6cb9-f80f-4370-913b-9abbfab54e18 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:47.815687 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c6cb9-f80f-4370-913b-9abbfab54e18 None admin] RBAC: Adding query filter params (interface=public, service_id=19cc88c7c25944da8770703e6c748931, region_id=RegionOne) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:47.815991 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c6cb9-f80f-4370-913b-9abbfab54e18 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:47.817343 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c6cb9-f80f-4370-913b-9abbfab54e18 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:47.822118 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 655/1300] 10.222.0.18 () {56 vars in 1251 bytes} [Thu Sep 1 16:24:47 2022] GET /identity/v3/endpoints?service_id=19cc88c7c25944da8770703e6c748931&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:47.827621 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29366d2-d507-42dd-8251-81f29ac220e5 None admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:47.912671 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:47.935293 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29366d2-d507-42dd-8251-81f29ac220e5 None admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:47.974063 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c61d9-a83a-4872-8f70-ce6f7a54f244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:47.974707 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c61d9-a83a-4872-8f70-ce6f7a54f244 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:47.974984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c61d9-a83a-4872-8f70-ce6f7a54f244 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:47.975254 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c61d9-a83a-4872-8f70-ce6f7a54f244 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:47.975855 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c61d9-a83a-4872-8f70-ce6f7a54f244 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:47.977199 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c61d9-a83a-4872-8f70-ce6f7a54f244 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:47.983847 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 646/1301] 10.222.0.18 () {56 vars in 1087 bytes} [Thu Sep 1 16:24:47 2022] GET /identity/v3/services => generated 2191 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:24:49.874483 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0088e5-b0ee-42fb-857d-8079901674d1 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:49.875048 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0088e5-b0ee-42fb-857d-8079901674d1 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:49.875165 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0088e5-b0ee-42fb-857d-8079901674d1 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:49.876607 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 656/1302] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:49.882118 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5dd67-40f8-48ac-ab83-6608f97dda5b None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:49.882524 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5dd67-40f8-48ac-ab83-6608f97dda5b None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:49.882731 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5dd67-40f8-48ac-ab83-6608f97dda5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:49.901062 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-54d5dd67-40f8-48ac-ab83-6608f97dda5b None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:49.902256 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-54d5dd67-40f8-48ac-ab83-6608f97dda5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:49.966328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 647/1303] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:49 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:50.228620 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0088e5-b0ee-42fb-857d-8079901674d1 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:50.325407 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:50.336002 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0088e5-b0ee-42fb-857d-8079901674d1 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:50.369436 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303e47f-1f8a-41f6-bf19-88a6cea3d67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:50.370058 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303e47f-1f8a-41f6-bf19-88a6cea3d67d None admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:50.370369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303e47f-1f8a-41f6-bf19-88a6cea3d67d None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:50.370645 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303e47f-1f8a-41f6-bf19-88a6cea3d67d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:50.371219 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303e47f-1f8a-41f6-bf19-88a6cea3d67d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:50.372574 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303e47f-1f8a-41f6-bf19-88a6cea3d67d None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:50.381709 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application [None req-a303e47f-1f8a-41f6-bf19-88a6cea3d67d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:24:50.381709 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:50.381709 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:24:50.381709 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:24:50.381709 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:24:50.381709 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:24:50.381709 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:24:50.381709 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:50.381709 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:24:50.381709 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:24:50.381709 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:24:50.381709 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:24:50.381709 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:50.381709 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:50.381709 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:50.381709 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:50.381709 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:50.382916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:50.382916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:50.382916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:50.382916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:50.382916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:50.382916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:50.382916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:24:50.382916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:24:50.382916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:24:50.382916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:24:50.382916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:50.382916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:50.382916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:50.382916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:50.382916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:50.382916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:50.383938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:50.383938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:50.383938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:50.383938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:50.383938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:50.383938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:50.383938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:50.383938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:50.383938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:50.383938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:50.383938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:24:50.383938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:50.383938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:24:50.383938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: ERROR keystone.server.flask.application Sep 01 16:24:50.383938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 657/1304] 10.222.0.18 () {56 vars in 1117 bytes} [Thu Sep 1 16:24:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:50.388845 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5dd67-40f8-48ac-ab83-6608f97dda5b None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:50.463667 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:50.487218 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5dd67-40f8-48ac-ab83-6608f97dda5b None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:50.520185 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890211a1-13bb-4e5d-8196-9da4a791f5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:50.520786 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890211a1-13bb-4e5d-8196-9da4a791f5f4 None admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:50.521068 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890211a1-13bb-4e5d-8196-9da4a791f5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:50.521343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890211a1-13bb-4e5d-8196-9da4a791f5f4 None admin] PATH_INFO: `/v3/services` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:50.521963 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890211a1-13bb-4e5d-8196-9da4a791f5f4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:50.522295 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890211a1-13bb-4e5d-8196-9da4a791f5f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:50.523645 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890211a1-13bb-4e5d-8196-9da4a791f5f4 None admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:50.528620 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 648/1305] 10.222.0.18 () {56 vars in 1116 bytes} [Thu Sep 1 16:24:50 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:50.534326 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303e47f-1f8a-41f6-bf19-88a6cea3d67d None admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:50.631437 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303e47f-1f8a-41f6-bf19-88a6cea3d67d None admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:50.663570 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dea7e5-d356-43e5-9875-c596ee4ada11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:50.664161 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dea7e5-d356-43e5-9875-c596ee4ada11 None admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:50.664442 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dea7e5-d356-43e5-9875-c596ee4ada11 None admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:50.664712 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dea7e5-d356-43e5-9875-c596ee4ada11 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:50.665377 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dea7e5-d356-43e5-9875-c596ee4ada11 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:50.666767 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dea7e5-d356-43e5-9875-c596ee4ada11 None admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:50.685714 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 658/1306] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:24:50 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:25:00.525641 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=30 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:00.682127 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:54.274612 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674b419-f09b-40e0-8fb3-0b10f22eaa2f None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:54.275059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674b419-f09b-40e0-8fb3-0b10f22eaa2f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:54.275277 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674b419-f09b-40e0-8fb3-0b10f22eaa2f None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:54.276523 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 649/1307] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:25:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:54.288268 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb811780-78c8-4829-b1e4-5556a4e3e552 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:54.290008 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb811780-78c8-4829-b1e4-5556a4e3e552 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:54.291234 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb811780-78c8-4829-b1e4-5556a4e3e552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:54.417094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-fb811780-78c8-4829-b1e4-5556a4e3e552 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:54.418585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-fb811780-78c8-4829-b1e4-5556a4e3e552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:54.558406 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 659/1308] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:54.848493 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38093022-991c-46cf-b4bc-d9a541a989f1 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:54.848776 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38093022-991c-46cf-b4bc-d9a541a989f1 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:54.849208 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38093022-991c-46cf-b4bc-d9a541a989f1 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:54.850888 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 650/1309] 10.222.0.18 () {54 vars in 965 bytes} [Thu Sep 1 16:25:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:54.856950 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20efc78-a9a8-4dd6-86b1-ee3411a3b6b8 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:54.857245 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20efc78-a9a8-4dd6-86b1-ee3411a3b6b8 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:54.857596 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20efc78-a9a8-4dd6-86b1-ee3411a3b6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:54.939860 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-c20efc78-a9a8-4dd6-86b1-ee3411a3b6b8 None None] MFA Rules not processed for user `ffa0f2160a934906b59e6f845edb0776`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:54.941135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-c20efc78-a9a8-4dd6-86b1-ee3411a3b6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:54.962277 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:55.020408 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 660/1310] 10.222.0.18 () {58 vars in 1062 bytes} [Thu Sep 1 16:25:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:55.029309 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38093022-991c-46cf-b4bc-d9a541a989f1 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:55.031357 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-38093022-991c-46cf-b4bc-d9a541a989f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:55.150300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38093022-991c-46cf-b4bc-d9a541a989f1 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:55.237293 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22337e00-6026-46d0-ad53-4aa9b28d7e09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:55.237906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22337e00-6026-46d0-ad53-4aa9b28d7e09 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:55.238186 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22337e00-6026-46d0-ad53-4aa9b28d7e09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:55.238486 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22337e00-6026-46d0-ad53-4aa9b28d7e09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:55.240072 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-22337e00-6026-46d0-ad53-4aa9b28d7e09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:55.303383 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:55.373412 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22337e00-6026-46d0-ad53-4aa9b28d7e09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:55.375214 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22337e00-6026-46d0-ad53-4aa9b28d7e09 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:55.397166 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 651/1311] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:25:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:25:57.594274 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742124ec-675f-4071-a6dd-3ed9d089253b None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:57.594843 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742124ec-675f-4071-a6dd-3ed9d089253b None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:57.594843 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742124ec-675f-4071-a6dd-3ed9d089253b None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:57.600461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 661/1312] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:25:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:57.601808 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932bc43-f541-4a31-9181-d61b97d6bdc8 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:57.602319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932bc43-f541-4a31-9181-d61b97d6bdc8 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:57.602428 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932bc43-f541-4a31-9181-d61b97d6bdc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:57.696425 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-9932bc43-f541-4a31-9181-d61b97d6bdc8 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:57.697746 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9932bc43-f541-4a31-9181-d61b97d6bdc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:57.815923 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 652/1313] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:25:58.097288 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742124ec-675f-4071-a6dd-3ed9d089253b None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:58.098867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-742124ec-675f-4071-a6dd-3ed9d089253b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:58.202206 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742124ec-675f-4071-a6dd-3ed9d089253b None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:58.273583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:58.276614 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee992f0-2564-4f22-bdc3-fe4de7d146de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:58.277483 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee992f0-2564-4f22-bdc3-fe4de7d146de service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:58.277808 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee992f0-2564-4f22-bdc3-fe4de7d146de service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:58.278311 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee992f0-2564-4f22-bdc3-fe4de7d146de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:58.280166 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-0ee992f0-2564-4f22-bdc3-fe4de7d146de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:58.394341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee992f0-2564-4f22-bdc3-fe4de7d146de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:58.395813 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee992f0-2564-4f22-bdc3-fe4de7d146de service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:58.414587 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 662/1314] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:25:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:00.637059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cfe58-fef7-4fd2-89ab-e02762eb2330 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:00.637562 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cfe58-fef7-4fd2-89ab-e02762eb2330 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:00.637562 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cfe58-fef7-4fd2-89ab-e02762eb2330 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:00.638914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 653/1315] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:00.644542 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d9555-6d8a-4783-b6ab-af6a080486b7 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:00.644928 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d9555-6d8a-4783-b6ab-af6a080486b7 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:00.645100 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d9555-6d8a-4783-b6ab-af6a080486b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:00.716550 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-2b0d9555-6d8a-4783-b6ab-af6a080486b7 None None] MFA Rules not processed for user `6d995078a79f4035bcbc04685a0579e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:00.717761 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-2b0d9555-6d8a-4783-b6ab-af6a080486b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:00.775766 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:00.820053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 663/1316] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:26:00 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:01.104289 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22cfe58-fef7-4fd2-89ab-e02762eb2330 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:01.105878 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-e22cfe58-fef7-4fd2-89ab-e02762eb2330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:01.116399 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:01.207809 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22cfe58-fef7-4fd2-89ab-e02762eb2330 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:01.278923 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db895883-87eb-4498-a11d-16a497ef227c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:01.279525 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db895883-87eb-4498-a11d-16a497ef227c service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:01.279807 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db895883-87eb-4498-a11d-16a497ef227c service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:01.280082 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db895883-87eb-4498-a11d-16a497ef227c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:01.282167 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-db895883-87eb-4498-a11d-16a497ef227c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:01.342928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:01.390574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db895883-87eb-4498-a11d-16a497ef227c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:01.392006 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db895883-87eb-4498-a11d-16a497ef227c service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:01.410070 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 654/1317] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:01 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:03.828833 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad4434-bda5-42a4-a5b6-b9f1f2087175 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:03.829362 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad4434-bda5-42a4-a5b6-b9f1f2087175 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:03.829362 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad4434-bda5-42a4-a5b6-b9f1f2087175 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:03.830769 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 664/1318] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:26:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:03.835864 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c993f6-4914-4b56-88ce-e19199a4a2b2 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:03.836295 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c993f6-4914-4b56-88ce-e19199a4a2b2 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:03.836422 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c993f6-4914-4b56-88ce-e19199a4a2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:03.902419 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-37c993f6-4914-4b56-88ce-e19199a4a2b2 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:03.903616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-37c993f6-4914-4b56-88ce-e19199a4a2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:03.998565 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 655/1319] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:26:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:04.288136 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddad4434-bda5-42a4-a5b6-b9f1f2087175 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:04.289670 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-ddad4434-bda5-42a4-a5b6-b9f1f2087175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:04.392739 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddad4434-bda5-42a4-a5b6-b9f1f2087175 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:04.466269 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e9c36-6983-45d9-96be-a94d88e31d9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:04.466864 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e9c36-6983-45d9-96be-a94d88e31d9e service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:04.467147 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e9c36-6983-45d9-96be-a94d88e31d9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:04.467421 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e9c36-6983-45d9-96be-a94d88e31d9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:04.469511 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-8f4e9c36-6983-45d9-96be-a94d88e31d9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:04.476339 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:04.575333 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e9c36-6983-45d9-96be-a94d88e31d9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:04.576771 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e9c36-6983-45d9-96be-a94d88e31d9e service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:04.595775 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 665/1320] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:07.143696 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959bc94-e218-43ff-a86d-9095cc74e742 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:07.144238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959bc94-e218-43ff-a86d-9095cc74e742 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:07.144238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959bc94-e218-43ff-a86d-9095cc74e742 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:07.145589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 656/1321] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:07.150796 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593014e-062a-4bb9-aba3-052e3a79dbbb None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:07.151228 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593014e-062a-4bb9-aba3-052e3a79dbbb None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:07.151361 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593014e-062a-4bb9-aba3-052e3a79dbbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:07.217012 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-0593014e-062a-4bb9-aba3-052e3a79dbbb None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:07.218740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-0593014e-062a-4bb9-aba3-052e3a79dbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:07.276556 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:07.316339 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 666/1322] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:26:07.620592 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908753d-880d-4b1b-a786-57545a1be324 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:07.620883 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908753d-880d-4b1b-a786-57545a1be324 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:07.621162 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908753d-880d-4b1b-a786-57545a1be324 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:07.622549 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 657/1323] 10.222.0.18 () {54 vars in 965 bytes} [Thu Sep 1 16:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:07.628647 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e1320-dee5-4637-883d-13180856436a None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:07.628941 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e1320-dee5-4637-883d-13180856436a None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:07.629221 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e1320-dee5-4637-883d-13180856436a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:07.705651 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-e38e1320-dee5-4637-883d-13180856436a None None] MFA Rules not processed for user `ffa0f2160a934906b59e6f845edb0776`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:07.706892 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-e38e1320-dee5-4637-883d-13180856436a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:07.785443 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 667/1324] 10.222.0.18 () {58 vars in 1062 bytes} [Thu Sep 1 16:26:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:07.794388 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2908753d-880d-4b1b-a786-57545a1be324 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:07.796475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2908753d-880d-4b1b-a786-57545a1be324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:07.817138 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:07.903888 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2908753d-880d-4b1b-a786-57545a1be324 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:07.978503 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab7a5f-dc2c-445f-afaf-ad6eabfe4981 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:07.979127 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab7a5f-dc2c-445f-afaf-ad6eabfe4981 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:07.979412 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab7a5f-dc2c-445f-afaf-ad6eabfe4981 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:07.979685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab7a5f-dc2c-445f-afaf-ad6eabfe4981 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:07.981276 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-30ab7a5f-dc2c-445f-afaf-ad6eabfe4981 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:08.062921 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:08.105937 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ab7a5f-dc2c-445f-afaf-ad6eabfe4981 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:08.107437 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ab7a5f-dc2c-445f-afaf-ad6eabfe4981 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:08.127089 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 658/1325] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:10.326659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0cf8d9-8381-4770-a2cd-3117f13958af None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:10.327717 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0cf8d9-8381-4770-a2cd-3117f13958af None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:10.327717 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0cf8d9-8381-4770-a2cd-3117f13958af None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:10.329083 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 668/1326] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:26:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:10.334225 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043127e-1667-4028-beb8-f2e1083c45a4 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:10.334658 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043127e-1667-4028-beb8-f2e1083c45a4 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:10.334910 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043127e-1667-4028-beb8-f2e1083c45a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:10.401851 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-0043127e-1667-4028-beb8-f2e1083c45a4 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:10.403123 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0043127e-1667-4028-beb8-f2e1083c45a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:10.506016 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 659/1327] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:26:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:10.794389 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0cf8d9-8381-4770-a2cd-3117f13958af None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:10.795951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-df0cf8d9-8381-4770-a2cd-3117f13958af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:10.856917 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:10.901255 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0cf8d9-8381-4770-a2cd-3117f13958af None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:10.974803 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78651d7-f80d-47a7-8feb-71cb0807e7a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:10.975419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78651d7-f80d-47a7-8feb-71cb0807e7a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:10.975703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78651d7-f80d-47a7-8feb-71cb0807e7a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:10.975981 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78651d7-f80d-47a7-8feb-71cb0807e7a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:10.977569 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-a78651d7-f80d-47a7-8feb-71cb0807e7a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:11.083861 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:11.086531 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78651d7-f80d-47a7-8feb-71cb0807e7a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:11.088286 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78651d7-f80d-47a7-8feb-71cb0807e7a4 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:11.106874 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 669/1328] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:13.116220 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9244c-6bff-4507-81a3-b10d3707915f None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:13.116786 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9244c-6bff-4507-81a3-b10d3707915f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:13.116786 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9244c-6bff-4507-81a3-b10d3707915f None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:13.119242 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 660/1329] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:26:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:13.123557 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73842f-aa8d-4789-bc17-c296a728a571 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:13.124074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73842f-aa8d-4789-bc17-c296a728a571 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:13.124074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73842f-aa8d-4789-bc17-c296a728a571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:13.192440 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-1a73842f-aa8d-4789-bc17-c296a728a571 None None] MFA Rules not processed for user `6d995078a79f4035bcbc04685a0579e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:13.193652 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-1a73842f-aa8d-4789-bc17-c296a728a571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:13.288170 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 670/1330] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:26:13 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:13.567203 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9244c-6bff-4507-81a3-b10d3707915f None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:13.568747 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-14c9244c-6bff-4507-81a3-b10d3707915f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:13.600378 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:13.673987 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9244c-6bff-4507-81a3-b10d3707915f None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:13.750274 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a69fe4-c490-4af4-9b09-4dc6d2a6ff41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:13.750915 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a69fe4-c490-4af4-9b09-4dc6d2a6ff41 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:13.751191 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a69fe4-c490-4af4-9b09-4dc6d2a6ff41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:13.751462 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a69fe4-c490-4af4-9b09-4dc6d2a6ff41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:13.753048 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-10a69fe4-c490-4af4-9b09-4dc6d2a6ff41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:13.832594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:13.866635 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a69fe4-c490-4af4-9b09-4dc6d2a6ff41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:13.868093 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a69fe4-c490-4af4-9b09-4dc6d2a6ff41 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:13.887510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 661/1331] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:13 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:16.011441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d809767e-cd18-468a-be5a-4b1976655b5c None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:16.012007 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d809767e-cd18-468a-be5a-4b1976655b5c None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:16.012007 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d809767e-cd18-468a-be5a-4b1976655b5c None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:16.013356 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 671/1332] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:26:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:16.018955 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df29e8f-10a7-4f08-97ed-a5e4a9615126 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:16.019362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df29e8f-10a7-4f08-97ed-a5e4a9615126 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:16.019544 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df29e8f-10a7-4f08-97ed-a5e4a9615126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:16.087479 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-1df29e8f-10a7-4f08-97ed-a5e4a9615126 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:16.088691 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-1df29e8f-10a7-4f08-97ed-a5e4a9615126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:16.187416 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 662/1333] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:26:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:16.467068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d809767e-cd18-468a-be5a-4b1976655b5c None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:16.468608 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-d809767e-cd18-468a-be5a-4b1976655b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:16.536753 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:16.570848 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d809767e-cd18-468a-be5a-4b1976655b5c None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:16.641642 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a96c45-2cf0-4b62-80ed-a801273f89e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:16.642175 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a96c45-2cf0-4b62-80ed-a801273f89e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:16.642480 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a96c45-2cf0-4b62-80ed-a801273f89e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:16.642749 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a96c45-2cf0-4b62-80ed-a801273f89e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:16.644308 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-d8a96c45-2cf0-4b62-80ed-a801273f89e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:16.750980 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a96c45-2cf0-4b62-80ed-a801273f89e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:16.752431 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a96c45-2cf0-4b62-80ed-a801273f89e5 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:16.762353 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:16.771487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 672/1334] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:18.741181 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2a609-61d1-4c49-aaa4-f21e7d5fc0df None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:18.741704 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2a609-61d1-4c49-aaa4-f21e7d5fc0df None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:18.741704 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2a609-61d1-4c49-aaa4-f21e7d5fc0df None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:18.743078 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 663/1335] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:18.748032 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65396101-430d-423f-85d8-f15e480bc06e None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:18.748435 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65396101-430d-423f-85d8-f15e480bc06e None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:18.748615 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65396101-430d-423f-85d8-f15e480bc06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:18.812208 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-65396101-430d-423f-85d8-f15e480bc06e None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:18.813419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-65396101-430d-423f-85d8-f15e480bc06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:18.907641 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 673/1336] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:19.187514 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2a609-61d1-4c49-aaa4-f21e7d5fc0df None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:19.189047 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-efa2a609-61d1-4c49-aaa4-f21e7d5fc0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:19.229123 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:19.292338 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2a609-61d1-4c49-aaa4-f21e7d5fc0df None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:19.365369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ed6d6-aa0a-4634-b47c-15c732e2a62c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:19.365986 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ed6d6-aa0a-4634-b47c-15c732e2a62c service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:19.366290 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ed6d6-aa0a-4634-b47c-15c732e2a62c service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:19.366569 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ed6d6-aa0a-4634-b47c-15c732e2a62c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:19.368164 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-171ed6d6-aa0a-4634-b47c-15c732e2a62c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:19.450253 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:19.475982 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ed6d6-aa0a-4634-b47c-15c732e2a62c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:19.477420 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ed6d6-aa0a-4634-b47c-15c732e2a62c service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:19.495738 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 664/1337] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:21.716357 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2bcc6-5999-42ce-b5d9-b06c7b15d71f None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:21.716894 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2bcc6-5999-42ce-b5d9-b06c7b15d71f None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:21.716988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2bcc6-5999-42ce-b5d9-b06c7b15d71f None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:21.718334 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 674/1338] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:21.723472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47409513-8535-4231-9b5e-4c3f60d417ba None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:21.723867 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47409513-8535-4231-9b5e-4c3f60d417ba None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:21.724019 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47409513-8535-4231-9b5e-4c3f60d417ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:21.789291 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-47409513-8535-4231-9b5e-4c3f60d417ba None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:21.790525 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-47409513-8535-4231-9b5e-4c3f60d417ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:21.887582 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 665/1339] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:26:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:22.165571 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b2bcc6-5999-42ce-b5d9-b06c7b15d71f None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:22.167134 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-98b2bcc6-5999-42ce-b5d9-b06c7b15d71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:22.242247 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:22.269255 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b2bcc6-5999-42ce-b5d9-b06c7b15d71f None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:22.340555 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a81eb08-4a33-449b-8ca0-f152eda09c0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:22.341095 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81eb08-4a33-449b-8ca0-f152eda09c0f service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:22.341404 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81eb08-4a33-449b-8ca0-f152eda09c0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:22.341678 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81eb08-4a33-449b-8ca0-f152eda09c0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:22.343769 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-7a81eb08-4a33-449b-8ca0-f152eda09c0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:22.448612 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a81eb08-4a33-449b-8ca0-f152eda09c0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:22.450046 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a81eb08-4a33-449b-8ca0-f152eda09c0f service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:22.468386 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 675/1340] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:26.271262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08629c-1576-47c8-bb9f-e2f07eb33ed9 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:26.271697 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08629c-1576-47c8-bb9f-e2f07eb33ed9 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:26.272050 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08629c-1576-47c8-bb9f-e2f07eb33ed9 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:26.273901 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 666/1341] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:26:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:26.278810 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41eace0-7af1-41a1-bb6d-7dd9edd9f955 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:26.279309 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41eace0-7af1-41a1-bb6d-7dd9edd9f955 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:26.279411 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41eace0-7af1-41a1-bb6d-7dd9edd9f955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:26.284787 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:26.345467 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-d41eace0-7af1-41a1-bb6d-7dd9edd9f955 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:26.346733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-d41eace0-7af1-41a1-bb6d-7dd9edd9f955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:26.451164 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 676/1342] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:26.731536 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08629c-1576-47c8-bb9f-e2f07eb33ed9 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:26.733644 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0e08629c-1576-47c8-bb9f-e2f07eb33ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:26.784671 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:26.841025 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08629c-1576-47c8-bb9f-e2f07eb33ed9 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:26.925418 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65edaa-db49-4cb3-89d5-78c1413a9aa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:26.926043 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65edaa-db49-4cb3-89d5-78c1413a9aa4 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:26.926358 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65edaa-db49-4cb3-89d5-78c1413a9aa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:26.926638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65edaa-db49-4cb3-89d5-78c1413a9aa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:26.928253 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-db65edaa-db49-4cb3-89d5-78c1413a9aa4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:27.024840 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:27.041585 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db65edaa-db49-4cb3-89d5-78c1413a9aa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:27.043091 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db65edaa-db49-4cb3-89d5-78c1413a9aa4 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:27.060914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 667/1343] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:29.592570 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8b3e2-c03c-499a-90d5-cf1245b190a1 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:29.593121 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8b3e2-c03c-499a-90d5-cf1245b190a1 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:29.593121 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8b3e2-c03c-499a-90d5-cf1245b190a1 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:29.594510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 677/1344] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:26:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:29.599879 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48527ed6-c584-42ee-8071-ccd31532b748 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:29.600302 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48527ed6-c584-42ee-8071-ccd31532b748 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:29.600431 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48527ed6-c584-42ee-8071-ccd31532b748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:29.666879 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-48527ed6-c584-42ee-8071-ccd31532b748 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:29.668009 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-48527ed6-c584-42ee-8071-ccd31532b748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:29.763367 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 668/1345] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:26:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:30.051273 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e8b3e2-c03c-499a-90d5-cf1245b190a1 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:30.052810 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-55e8b3e2-c03c-499a-90d5-cf1245b190a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:30.140068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:30.158149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e8b3e2-c03c-499a-90d5-cf1245b190a1 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:30.236845 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d148c50-2960-4b41-a6ee-2eb617b5d72b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:30.237452 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d148c50-2960-4b41-a6ee-2eb617b5d72b service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:30.237740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d148c50-2960-4b41-a6ee-2eb617b5d72b service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:30.238011 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d148c50-2960-4b41-a6ee-2eb617b5d72b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:30.239650 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-2d148c50-2960-4b41-a6ee-2eb617b5d72b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:30.347222 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d148c50-2960-4b41-a6ee-2eb617b5d72b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:30.348607 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d148c50-2960-4b41-a6ee-2eb617b5d72b service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:30.368733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 678/1346] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:34.249965 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31906636-a267-4722-8a19-04e509e8e71c None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:34.250451 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31906636-a267-4722-8a19-04e509e8e71c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:34.250615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31906636-a267-4722-8a19-04e509e8e71c None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:34.252357 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 669/1347] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:34.257863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409cea4c-fa4c-478a-8ed0-ef7d0e7718c7 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:34.258408 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409cea4c-fa4c-478a-8ed0-ef7d0e7718c7 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:34.258503 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409cea4c-fa4c-478a-8ed0-ef7d0e7718c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:34.275776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:34.334935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-409cea4c-fa4c-478a-8ed0-ef7d0e7718c7 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:34.336359 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-409cea4c-fa4c-478a-8ed0-ef7d0e7718c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:34.456768 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 679/1348] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:26:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:34.737254 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31906636-a267-4722-8a19-04e509e8e71c None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:34.738849 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-31906636-a267-4722-8a19-04e509e8e71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:34.799723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:34.847404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31906636-a267-4722-8a19-04e509e8e71c None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:34.920405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aae391-aa21-46a1-b71b-1f3415fd3cef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:34.921015 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aae391-aa21-46a1-b71b-1f3415fd3cef service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:34.921299 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aae391-aa21-46a1-b71b-1f3415fd3cef service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:34.921573 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aae391-aa21-46a1-b71b-1f3415fd3cef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:34.923200 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b4aae391-aa21-46a1-b71b-1f3415fd3cef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:35.027243 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:35.031230 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aae391-aa21-46a1-b71b-1f3415fd3cef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:35.033215 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aae391-aa21-46a1-b71b-1f3415fd3cef service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:35.051556 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 670/1349] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:39.931876 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f283f2-c677-4691-b53c-9a2580fd5401 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:39.932774 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f283f2-c677-4691-b53c-9a2580fd5401 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:39.933265 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f283f2-c677-4691-b53c-9a2580fd5401 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:39.935657 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 680/1350] 10.222.0.18 () {54 vars in 921 bytes} [Thu Sep 1 16:26:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:39.941026 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85435a76-fa7e-47d8-a363-4281b8019dab None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:39.941468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85435a76-fa7e-47d8-a363-4281b8019dab None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:39.941646 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85435a76-fa7e-47d8-a363-4281b8019dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:40.073644 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-85435a76-fa7e-47d8-a363-4281b8019dab None None] MFA Rules not processed for user `3ddfb151bbf844efae73ec1990e7eaa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:40.074973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-85435a76-fa7e-47d8-a363-4281b8019dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:40.196344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 671/1351] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Sep 1 16:26:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:44.213997 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d1c5a-61dc-4719-a92d-86f3ba94cc7c None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:44.214984 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d1c5a-61dc-4719-a92d-86f3ba94cc7c None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:44.215374 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d1c5a-61dc-4719-a92d-86f3ba94cc7c None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:44.217492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 681/1352] 10.222.0.18 () {54 vars in 921 bytes} [Thu Sep 1 16:26:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:44.222220 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d556f9d-681e-40f6-8d13-915153cd43d8 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:44.222759 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d556f9d-681e-40f6-8d13-915153cd43d8 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:44.223024 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d556f9d-681e-40f6-8d13-915153cd43d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:44.283550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:44.309089 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-7d556f9d-681e-40f6-8d13-915153cd43d8 None None] MFA Rules not processed for user `3ddfb151bbf844efae73ec1990e7eaa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:44.310916 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7d556f9d-681e-40f6-8d13-915153cd43d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:44.435797 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 672/1353] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Sep 1 16:26:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:44.453949 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=59 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:45.899395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb32c1-2751-431c-8633-4d2a31bb250f None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:45.900056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb32c1-2751-431c-8633-4d2a31bb250f None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:45.900056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb32c1-2751-431c-8633-4d2a31bb250f None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:45.901506 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 682/1354] 10.222.0.18 () {54 vars in 921 bytes} [Thu Sep 1 16:26:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:45.906685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed3818-f310-40c0-9f45-b5d18cbacb0d None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:45.907121 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed3818-f310-40c0-9f45-b5d18cbacb0d None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:45.907240 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed3818-f310-40c0-9f45-b5d18cbacb0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:45.984488 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-31ed3818-f310-40c0-9f45-b5d18cbacb0d None None] MFA Rules not processed for user `3ddfb151bbf844efae73ec1990e7eaa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:45.985686 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-31ed3818-f310-40c0-9f45-b5d18cbacb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:46.028963 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:46.100407 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 673/1355] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Sep 1 16:26:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:49.298744 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be27daf-4cde-4524-a884-8be0d79d851e None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:49.299743 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be27daf-4cde-4524-a884-8be0d79d851e None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:49.300249 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be27daf-4cde-4524-a884-8be0d79d851e None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:49.302060 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 683/1356] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:26:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:49.307657 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6da9da-9312-4b57-80f2-1a5307fade04 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:49.308132 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6da9da-9312-4b57-80f2-1a5307fade04 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:49.308356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6da9da-9312-4b57-80f2-1a5307fade04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:49.394849 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-4a6da9da-9312-4b57-80f2-1a5307fade04 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:49.396284 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-4a6da9da-9312-4b57-80f2-1a5307fade04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:49.523335 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 674/1357] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:49.825473 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198ae91-1463-43b6-968e-8f01dc7328d9 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:49.825757 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198ae91-1463-43b6-968e-8f01dc7328d9 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:49.826186 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198ae91-1463-43b6-968e-8f01dc7328d9 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:49.827707 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 684/1358] 10.222.0.18 () {54 vars in 962 bytes} [Thu Sep 1 16:26:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:49.837263 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ae35f-b076-420f-b2e3-f8dc63cd63a2 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:49.838099 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ae35f-b076-420f-b2e3-f8dc63cd63a2 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:49.838765 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ae35f-b076-420f-b2e3-f8dc63cd63a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:49.861757 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:49.926806 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-9c2ae35f-b076-420f-b2e3-f8dc63cd63a2 None None] MFA Rules not processed for user `278e6d6aed414c758fe3e9f4df6bf81e`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:49.928145 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9c2ae35f-b076-420f-b2e3-f8dc63cd63a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:50.061813 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 675/1359] 10.222.0.18 () {58 vars in 1059 bytes} [Thu Sep 1 16:26:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:50.069664 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b198ae91-1463-43b6-968e-8f01dc7328d9 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:50.071295 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-b198ae91-1463-43b6-968e-8f01dc7328d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:50.223669 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b198ae91-1463-43b6-968e-8f01dc7328d9 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:50.311664 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:50.362454 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214cbfb5-4cc3-4109-8317-14fd980a4659 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:50.363424 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214cbfb5-4cc3-4109-8317-14fd980a4659 service nova] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:50.363910 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214cbfb5-4cc3-4109-8317-14fd980a4659 service nova] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:50.364349 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214cbfb5-4cc3-4109-8317-14fd980a4659 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:50.366104 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-214cbfb5-4cc3-4109-8317-14fd980a4659 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:50.480176 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214cbfb5-4cc3-4109-8317-14fd980a4659 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:50.482457 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214cbfb5-4cc3-4109-8317-14fd980a4659 service nova] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:50.512879 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 685/1360] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:52.658209 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71d80e-7b88-4a6f-b4f2-8a4fa8b5ba5d None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:52.659116 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71d80e-7b88-4a6f-b4f2-8a4fa8b5ba5d None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:52.659395 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71d80e-7b88-4a6f-b4f2-8a4fa8b5ba5d None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:52.660753 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 676/1361] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:26:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:52.666055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d0104-0b19-4a50-8a9a-fabdd9a7322e None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:52.666506 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d0104-0b19-4a50-8a9a-fabdd9a7322e None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:52.666647 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d0104-0b19-4a50-8a9a-fabdd9a7322e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:52.737053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-7a3d0104-0b19-4a50-8a9a-fabdd9a7322e None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:52.738271 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-7a3d0104-0b19-4a50-8a9a-fabdd9a7322e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:52.779289 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:52.855987 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 686/1362] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:53.146152 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e71d80e-7b88-4a6f-b4f2-8a4fa8b5ba5d None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:53.147892 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6e71d80e-7b88-4a6f-b4f2-8a4fa8b5ba5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:53.219164 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:53.309921 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e71d80e-7b88-4a6f-b4f2-8a4fa8b5ba5d None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:53.440259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e9b7e-6333-46ca-90bb-e9cdb444282b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:53.440885 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e9b7e-6333-46ca-90bb-e9cdb444282b service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:53.441244 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e9b7e-6333-46ca-90bb-e9cdb444282b service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:53.441560 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e9b7e-6333-46ca-90bb-e9cdb444282b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:53.443121 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-353e9b7e-6333-46ca-90bb-e9cdb444282b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:53.472087 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:53.550166 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e9b7e-6333-46ca-90bb-e9cdb444282b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:53.551753 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e9b7e-6333-46ca-90bb-e9cdb444282b service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:53.572735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 677/1363] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:55.829404 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1780e4-c9e1-44cf-b297-059c56ad4ff3 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:55.829973 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1780e4-c9e1-44cf-b297-059c56ad4ff3 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:55.829973 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1780e4-c9e1-44cf-b297-059c56ad4ff3 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:55.831327 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 687/1364] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:26:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:55.836897 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0b477-72aa-4d3c-9777-7693fd1e9035 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:55.837324 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0b477-72aa-4d3c-9777-7693fd1e9035 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:55.837471 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0b477-72aa-4d3c-9777-7693fd1e9035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:55.910571 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-9ff0b477-72aa-4d3c-9777-7693fd1e9035 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:55.911805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9ff0b477-72aa-4d3c-9777-7693fd1e9035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:56.011231 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:56.014335 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 678/1365] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:56.310828 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7263f9-3276-49fa-bd69-aa2c69c3d1c4 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:56.311113 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7263f9-3276-49fa-bd69-aa2c69c3d1c4 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:56.311566 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7263f9-3276-49fa-bd69-aa2c69c3d1c4 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:56.313051 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 688/1366] 10.222.0.18 () {54 vars in 962 bytes} [Thu Sep 1 16:26:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:56.318893 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e52e0-689d-43b0-857d-8f43d660550a None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:56.319287 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e52e0-689d-43b0-857d-8f43d660550a None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:56.319663 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e52e0-689d-43b0-857d-8f43d660550a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:56.424239 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-e88e52e0-689d-43b0-857d-8f43d660550a None None] MFA Rules not processed for user `278e6d6aed414c758fe3e9f4df6bf81e`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:56.425622 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-e88e52e0-689d-43b0-857d-8f43d660550a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:56.552887 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 679/1367] 10.222.0.18 () {58 vars in 1059 bytes} [Thu Sep 1 16:26:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:56.561239 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7263f9-3276-49fa-bd69-aa2c69c3d1c4 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:56.562875 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-4a7263f9-3276-49fa-bd69-aa2c69c3d1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:56.721823 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7263f9-3276-49fa-bd69-aa2c69c3d1c4 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:56.743845 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:56.842393 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1cf737-a628-4cad-8763-c3617ba0ae55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:56.843150 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1cf737-a628-4cad-8763-c3617ba0ae55 service nova] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:56.843544 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1cf737-a628-4cad-8763-c3617ba0ae55 service nova] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:56.843940 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1cf737-a628-4cad-8763-c3617ba0ae55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:56.845657 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-ea1cf737-a628-4cad-8763-c3617ba0ae55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:56.949088 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1cf737-a628-4cad-8763-c3617ba0ae55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:56.950796 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1cf737-a628-4cad-8763-c3617ba0ae55 service nova] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:56.971107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 689/1368] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:57.473728 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be705b-7240-47f4-9107-7ab2ef5fd195 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:57.474279 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be705b-7240-47f4-9107-7ab2ef5fd195 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:57.474474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be705b-7240-47f4-9107-7ab2ef5fd195 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:57.475828 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 680/1369] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:26:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:26:57.481002 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b07d4-d4b8-4912-9d77-722c1ff7e742 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:57.481302 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b07d4-d4b8-4912-9d77-722c1ff7e742 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:57.481585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b07d4-d4b8-4912-9d77-722c1ff7e742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:57.515450 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:57.551261 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-088b07d4-d4b8-4912-9d77-722c1ff7e742 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:57.552486 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-088b07d4-d4b8-4912-9d77-722c1ff7e742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:57.654133 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 690/1370] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:26:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:58.243520 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa40d0-fc5e-4b13-b349-e6461a6d12a4 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:58.243520 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa40d0-fc5e-4b13-b349-e6461a6d12a4 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:58.243520 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa40d0-fc5e-4b13-b349-e6461a6d12a4 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:58.245799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 681/1371] 10.222.0.18 () {54 vars in 964 bytes} [Thu Sep 1 16:26:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:58.250884 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138562e8-438e-4bed-a75b-1be1700bb806 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:58.251406 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138562e8-438e-4bed-a75b-1be1700bb806 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:58.251577 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138562e8-438e-4bed-a75b-1be1700bb806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:58.329594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-138562e8-438e-4bed-a75b-1be1700bb806 None None] MFA Rules not processed for user `b78747bb9140457896fe62d4f537592c`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:58.330815 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-138562e8-438e-4bed-a75b-1be1700bb806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:58.380851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:58.413606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 691/1372] 10.222.0.18 () {58 vars in 1061 bytes} [Thu Sep 1 16:26:58 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:58.421901 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa40d0-fc5e-4b13-b349-e6461a6d12a4 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:58.423435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-64aa40d0-fc5e-4b13-b349-e6461a6d12a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:58.475979 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:58.532508 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa40d0-fc5e-4b13-b349-e6461a6d12a4 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:58.612639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77268e-2167-4722-b73b-c49a82701d9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:58.613258 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77268e-2167-4722-b73b-c49a82701d9f service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:58.613533 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77268e-2167-4722-b73b-c49a82701d9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:58.613821 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77268e-2167-4722-b73b-c49a82701d9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:58.615469 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-1d77268e-2167-4722-b73b-c49a82701d9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:58.723195 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:58.736329 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77268e-2167-4722-b73b-c49a82701d9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:58.737766 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77268e-2167-4722-b73b-c49a82701d9f service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:58.756738 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 682/1373] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:58.977986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f24e69-b710-4df1-abb8-59e2eea2f3bf None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:58.978377 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f24e69-b710-4df1-abb8-59e2eea2f3bf None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:58.978715 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f24e69-b710-4df1-abb8-59e2eea2f3bf None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:58.980182 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 692/1374] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:26:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:58.985963 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606d674-a4d5-4197-aba2-b3bda0dc7c38 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:58.986300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606d674-a4d5-4197-aba2-b3bda0dc7c38 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:58.986804 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606d674-a4d5-4197-aba2-b3bda0dc7c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:59.054204 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-a606d674-a4d5-4197-aba2-b3bda0dc7c38 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:59.055446 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a606d674-a4d5-4197-aba2-b3bda0dc7c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:59.153711 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 683/1375] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:59.453337 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f24e69-b710-4df1-abb8-59e2eea2f3bf None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:59.456221 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-f9f24e69-b710-4df1-abb8-59e2eea2f3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:59.607211 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f24e69-b710-4df1-abb8-59e2eea2f3bf None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:59.657967 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:59.719104 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f569d4d0-b888-4939-9412-abd379b8e75b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:59.719633 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569d4d0-b888-4939-9412-abd379b8e75b service nova] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:59.719912 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569d4d0-b888-4939-9412-abd379b8e75b service nova] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:59.720183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569d4d0-b888-4939-9412-abd379b8e75b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:59.721756 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-f569d4d0-b888-4939-9412-abd379b8e75b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:59.809383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f569d4d0-b888-4939-9412-abd379b8e75b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:59.811052 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f569d4d0-b888-4939-9412-abd379b8e75b service nova] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:59.829087 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 693/1376] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:27:01.748312 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf020f-f900-4b48-9e7d-6822a24e93fe None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:01.748858 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf020f-f900-4b48-9e7d-6822a24e93fe None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:01.748858 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf020f-f900-4b48-9e7d-6822a24e93fe None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:01.750192 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 684/1377] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:27:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:01.758073 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7234760-6ec0-42c9-8050-1da56a9a75c4 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:01.758073 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7234760-6ec0-42c9-8050-1da56a9a75c4 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:01.758073 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7234760-6ec0-42c9-8050-1da56a9a75c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:01.837479 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-c7234760-6ec0-42c9-8050-1da56a9a75c4 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:01.838627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-c7234760-6ec0-42c9-8050-1da56a9a75c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:01.845768 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:01.938053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 694/1378] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:27:02.232564 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf020f-f900-4b48-9e7d-6822a24e93fe None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:02.234948 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2cbf020f-f900-4b48-9e7d-6822a24e93fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:02.309887 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:02.413336 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf020f-f900-4b48-9e7d-6822a24e93fe None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:02.539697 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd37dc-23e0-44c2-9a55-25621822cffe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:02.540441 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd37dc-23e0-44c2-9a55-25621822cffe service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:02.540790 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd37dc-23e0-44c2-9a55-25621822cffe service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:02.541130 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd37dc-23e0-44c2-9a55-25621822cffe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:02.543018 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-04dd37dc-23e0-44c2-9a55-25621822cffe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:02.562831 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:02.645143 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dd37dc-23e0-44c2-9a55-25621822cffe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:02.646943 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dd37dc-23e0-44c2-9a55-25621822cffe service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:02.666277 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 685/1379] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:27:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:27:04.599762 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd5092-c250-4162-9c93-48ec7aec1cfa None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:04.600298 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd5092-c250-4162-9c93-48ec7aec1cfa None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:04.600298 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd5092-c250-4162-9c93-48ec7aec1cfa None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:04.601807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 695/1380] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:27:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:04.607473 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b125e5e-5cdf-4518-8614-50666b67b97f None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:04.607899 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b125e5e-5cdf-4518-8614-50666b67b97f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:04.608015 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b125e5e-5cdf-4518-8614-50666b67b97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:04.677513 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-9b125e5e-5cdf-4518-8614-50666b67b97f None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:04.678677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9b125e5e-5cdf-4518-8614-50666b67b97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:04.774160 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:04.786548 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 686/1381] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:27:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:05.073087 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd5092-c250-4162-9c93-48ec7aec1cfa None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:05.074768 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-67bd5092-c250-4162-9c93-48ec7aec1cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:05.220797 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:05.232745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd5092-c250-4162-9c93-48ec7aec1cfa None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:05.362593 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd9453-c86e-4cd8-8826-8b7232195200 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:05.366190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd9453-c86e-4cd8-8826-8b7232195200 service nova] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:05.366190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd9453-c86e-4cd8-8826-8b7232195200 service nova] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:05.366190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd9453-c86e-4cd8-8826-8b7232195200 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:05.366190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-abbd9453-c86e-4cd8-8826-8b7232195200 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:05.460863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd9453-c86e-4cd8-8826-8b7232195200 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:05.464159 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd9453-c86e-4cd8-8826-8b7232195200 service nova] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:05.479843 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:05.490206 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 696/1382] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:27:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:27:07.468776 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367df7cb-7a15-4865-9e98-4882f53e605c None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:07.469304 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367df7cb-7a15-4865-9e98-4882f53e605c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:07.469391 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367df7cb-7a15-4865-9e98-4882f53e605c None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:07.470712 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 687/1383] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:27:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:07.477591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03195483-2991-4dec-9e5e-1214e6f05ee6 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:07.478025 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03195483-2991-4dec-9e5e-1214e6f05ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:07.478333 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03195483-2991-4dec-9e5e-1214e6f05ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:07.546709 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-03195483-2991-4dec-9e5e-1214e6f05ee6 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:07.547913 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-03195483-2991-4dec-9e5e-1214e6f05ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:07.648674 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 697/1384] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:27:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:07.944788 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367df7cb-7a15-4865-9e98-4882f53e605c None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:07.946335 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-367df7cb-7a15-4865-9e98-4882f53e605c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:08.105217 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367df7cb-7a15-4865-9e98-4882f53e605c None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:08.180017 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:08.234092 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc82ad2f-5dec-44aa-8ad0-f2fe67bc6e70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:08.234880 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82ad2f-5dec-44aa-8ad0-f2fe67bc6e70 service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:08.235159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82ad2f-5dec-44aa-8ad0-f2fe67bc6e70 service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:08.235469 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82ad2f-5dec-44aa-8ad0-f2fe67bc6e70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:08.237403 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-cc82ad2f-5dec-44aa-8ad0-f2fe67bc6e70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:08.334403 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc82ad2f-5dec-44aa-8ad0-f2fe67bc6e70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:08.336126 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc82ad2f-5dec-44aa-8ad0-f2fe67bc6e70 service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:08.359560 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 688/1385] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:27:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:27:10.255228 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a34930-1872-4f05-88a9-03a7dcb728db None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:10.255736 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a34930-1872-4f05-88a9-03a7dcb728db None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:10.255995 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a34930-1872-4f05-88a9-03a7dcb728db None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:10.257354 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 698/1386] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:10.263967 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198934c4-2062-4eef-9613-73f71d106217 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:10.264388 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198934c4-2062-4eef-9613-73f71d106217 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:10.264513 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198934c4-2062-4eef-9613-73f71d106217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:10.330642 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-198934c4-2062-4eef-9613-73f71d106217 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:10.331992 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-198934c4-2062-4eef-9613-73f71d106217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:10.372841 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:10.436652 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 689/1387] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:10.724306 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a34930-1872-4f05-88a9-03a7dcb728db None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:10.725900 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-48a34930-1872-4f05-88a9-03a7dcb728db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:10.810657 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:10.880376 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a34930-1872-4f05-88a9-03a7dcb728db None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:11.000889 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe4f5f-05dd-4526-897a-bccb0b4ab093 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:11.001415 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe4f5f-05dd-4526-897a-bccb0b4ab093 service nova] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:11.001733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe4f5f-05dd-4526-897a-bccb0b4ab093 service nova] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:11.002065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe4f5f-05dd-4526-897a-bccb0b4ab093 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:11.003996 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-c7fe4f5f-05dd-4526-897a-bccb0b4ab093 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:11.048405 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:11.100550 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fe4f5f-05dd-4526-897a-bccb0b4ab093 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:11.102198 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fe4f5f-05dd-4526-897a-bccb0b4ab093 service nova] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:11.121149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 699/1388] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:27:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:27:13.070167 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d3630-bdd7-42c3-9ab4-ac6d8aa375dc None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:13.070735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d3630-bdd7-42c3-9ab4-ac6d8aa375dc None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:13.070735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d3630-bdd7-42c3-9ab4-ac6d8aa375dc None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:13.072108 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 690/1389] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:27:13.077203 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d297d6-e40b-451b-bc33-a9df47305b47 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:13.077626 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d297d6-e40b-451b-bc33-a9df47305b47 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:13.077786 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d297d6-e40b-451b-bc33-a9df47305b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:13.143608 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-c7d297d6-e40b-451b-bc33-a9df47305b47 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:13.144834 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-c7d297d6-e40b-451b-bc33-a9df47305b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:13.241401 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 700/1390] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:13.535002 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d3630-bdd7-42c3-9ab4-ac6d8aa375dc None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:13.536525 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-780d3630-bdd7-42c3-9ab4-ac6d8aa375dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:13.688474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d3630-bdd7-42c3-9ab4-ac6d8aa375dc None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:13.706643 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:13.809797 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b098e98-7ea2-4feb-ab14-d8d8c07c0945 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:13.810437 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b098e98-7ea2-4feb-ab14-d8d8c07c0945 service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:13.810720 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b098e98-7ea2-4feb-ab14-d8d8c07c0945 service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:13.810993 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b098e98-7ea2-4feb-ab14-d8d8c07c0945 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:13.812602 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-1b098e98-7ea2-4feb-ab14-d8d8c07c0945 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:13.904995 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b098e98-7ea2-4feb-ab14-d8d8c07c0945 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:13.906694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b098e98-7ea2-4feb-ab14-d8d8c07c0945 service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:13.925634 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 691/1391] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:27:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:27:16.068156 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33369729-118e-4b03-b5ab-55b0f9a3ba34 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:16.068771 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33369729-118e-4b03-b5ab-55b0f9a3ba34 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:16.068771 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33369729-118e-4b03-b5ab-55b0f9a3ba34 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:16.070110 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 701/1392] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:16.075811 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa0dbe-32c1-4c1d-9def-4f9998c9249c None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:16.076235 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa0dbe-32c1-4c1d-9def-4f9998c9249c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:16.076344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa0dbe-32c1-4c1d-9def-4f9998c9249c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:16.102937 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:16.145051 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-dcfa0dbe-32c1-4c1d-9def-4f9998c9249c None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:16.146276 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-dcfa0dbe-32c1-4c1d-9def-4f9998c9249c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:16.240898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 692/1393] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:27:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:27:16.529512 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33369729-118e-4b03-b5ab-55b0f9a3ba34 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:16.531083 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-33369729-118e-4b03-b5ab-55b0f9a3ba34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:16.546362 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:16.681411 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33369729-118e-4b03-b5ab-55b0f9a3ba34 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:16.768439 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:16.795806 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4519f892-d40e-416e-a45d-2cb3f319244b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:16.796408 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519f892-d40e-416e-a45d-2cb3f319244b service nova] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:16.796687 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519f892-d40e-416e-a45d-2cb3f319244b service nova] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:16.796957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519f892-d40e-416e-a45d-2cb3f319244b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:16.798573 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-4519f892-d40e-416e-a45d-2cb3f319244b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:16.885486 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4519f892-d40e-416e-a45d-2cb3f319244b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:16.887184 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4519f892-d40e-416e-a45d-2cb3f319244b service nova] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:16.905700 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 702/1394] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:27:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:27:17.504108 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213a4e3-9319-4f19-b9c8-797f9048fdea None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:17.504690 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213a4e3-9319-4f19-b9c8-797f9048fdea None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:17.504690 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213a4e3-9319-4f19-b9c8-797f9048fdea None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:17.505987 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 693/1395] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:17.511399 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbb47a-1238-49ff-8735-59795ccb98a5 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:17.511811 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbb47a-1238-49ff-8735-59795ccb98a5 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:17.511958 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbb47a-1238-49ff-8735-59795ccb98a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:17.579290 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-ddbbb47a-1238-49ff-8735-59795ccb98a5 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:17.580483 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-ddbbb47a-1238-49ff-8735-59795ccb98a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:17.629639 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:17.678397 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 703/1396] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:27:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:17.987529 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0baa6-2d6b-4232-86a6-a0c7404d6515 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:17.987793 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0baa6-2d6b-4232-86a6-a0c7404d6515 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:17.988098 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0baa6-2d6b-4232-86a6-a0c7404d6515 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:17.989481 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 694/1397] 10.222.0.18 () {54 vars in 967 bytes} [Thu Sep 1 16:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:17.995204 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da469ff5-f796-4252-bf16-03d9497ca13e None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:17.995540 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da469ff5-f796-4252-bf16-03d9497ca13e None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:17.995820 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da469ff5-f796-4252-bf16-03d9497ca13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:18.075668 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-da469ff5-f796-4252-bf16-03d9497ca13e None None] MFA Rules not processed for user `b09f6d550c9142bf956b225bf5172e42`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:18.076898 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-da469ff5-f796-4252-bf16-03d9497ca13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:18.152449 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 704/1398] 10.222.0.18 () {58 vars in 1064 bytes} [Thu Sep 1 16:27:17 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:18.159874 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0baa6-2d6b-4232-86a6-a0c7404d6515 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:18.161376 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-bcb0baa6-2d6b-4232-86a6-a0c7404d6515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:18.263030 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:18.264641 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0baa6-2d6b-4232-86a6-a0c7404d6515 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:18.337405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1748c1-5482-4619-a48e-711d0105e0e4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b09f6d550c9142bf956b225bf5172e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:18.338008 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1748c1-5482-4619-a48e-711d0105e0e4 service glance] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:18.338314 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1748c1-5482-4619-a48e-711d0105e0e4 service glance] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:18.338586 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1748c1-5482-4619-a48e-711d0105e0e4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:18.340687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2b1748c1-5482-4619-a48e-711d0105e0e4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:18.454345 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1748c1-5482-4619-a48e-711d0105e0e4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:18.455821 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1748c1-5482-4619-a48e-711d0105e0e4 service glance] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:18.474536 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 695/1399] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:27:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:27:18.819097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b2577-66a0-45b9-b8d0-b401e0da6bd1 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:18.819630 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b2577-66a0-45b9-b8d0-b401e0da6bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:18.819630 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b2577-66a0-45b9-b8d0-b401e0da6bd1 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:18.821031 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 705/1400] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:27:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:18.826946 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4b768-1356-4e18-98fc-027609e81b1c None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:18.827408 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4b768-1356-4e18-98fc-027609e81b1c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:18.827657 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4b768-1356-4e18-98fc-027609e81b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:18.870174 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:18.905124 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-a9c4b768-1356-4e18-98fc-027609e81b1c None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:18.906452 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a9c4b768-1356-4e18-98fc-027609e81b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:19.004122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 696/1401] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:19.202974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df2f03-e585-4b61-9659-afdc0582405d None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:19.203262 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df2f03-e585-4b61-9659-afdc0582405d None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:19.203539 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df2f03-e585-4b61-9659-afdc0582405d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:19.263077 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:19.272493 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-e5df2f03-e585-4b61-9659-afdc0582405d None None] MFA Rules not processed for user `a5b3e53fd1314e17ad28c42a44e8eff3`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:19.273886 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-e5df2f03-e585-4b61-9659-afdc0582405d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:19.293861 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c4b768-1356-4e18-98fc-027609e81b1c None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:19.296434 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a9c4b768-1356-4e18-98fc-027609e81b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:19.369504 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 706/1402] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Sep 1 16:27:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:19.423065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3f7a4-c12a-470a-8866-ac663fc69c46 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:19.423646 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:19.424136 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3f7a4-c12a-470a-8866-ac663fc69c46 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:19.424610 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3f7a4-c12a-470a-8866-ac663fc69c46 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:19.426495 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 707/1403] 10.222.0.18 () {54 vars in 943 bytes} [Thu Sep 1 16:27:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:27:19.428881 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c4b768-1356-4e18-98fc-027609e81b1c None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:19.434356 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26875ca-d6bb-4b40-a0ff-686146b2397d None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:19.434648 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26875ca-d6bb-4b40-a0ff-686146b2397d None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:19.434924 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26875ca-d6bb-4b40-a0ff-686146b2397d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:19.525533 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-f26875ca-d6bb-4b40-a0ff-686146b2397d None None] MFA Rules not processed for user `8137f6b2ced040bd9752239c7e987f03`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:19.527348 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-f26875ca-d6bb-4b40-a0ff-686146b2397d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:19.561780 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391050b-3b25-4e65-b8cb-0618b285a9bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:19.562433 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391050b-3b25-4e65-b8cb-0618b285a9bf service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:19.562713 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391050b-3b25-4e65-b8cb-0618b285a9bf service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:19.562985 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391050b-3b25-4e65-b8cb-0618b285a9bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:19.564584 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-1391050b-3b25-4e65-b8cb-0618b285a9bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:19.608442 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:19.646737 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 708/1404] 10.222.0.18 () {58 vars in 1040 bytes} [Thu Sep 1 16:27:19 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:27:19.657147 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26875ca-d6bb-4b40-a0ff-686146b2397d None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:19.659845 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1391050b-3b25-4e65-b8cb-0618b285a9bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:19.661462 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1391050b-3b25-4e65-b8cb-0618b285a9bf service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:19.666862 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26875ca-d6bb-4b40-a0ff-686146b2397d None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:19.679725 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 697/1405] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:27:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:27:19.783915 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faceae2-cd4e-4e06-9849-2fa41ed56a3b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8137f6b2ced040bd9752239c7e987f03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:19.784576 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faceae2-cd4e-4e06-9849-2fa41ed56a3b service swift] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:19.784861 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faceae2-cd4e-4e06-9849-2fa41ed56a3b service swift] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:19.785138 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faceae2-cd4e-4e06-9849-2fa41ed56a3b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:19.786910 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-7faceae2-cd4e-4e06-9849-2fa41ed56a3b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:19.875746 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:19.878806 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faceae2-cd4e-4e06-9849-2fa41ed56a3b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:19.881096 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faceae2-cd4e-4e06-9849-2fa41ed56a3b service swift] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:19.890436 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 709/1406] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 1 16:27:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 16:27:21.627966 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e560665-fd1d-4093-9418-e2f5ee3809a4 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:21.628565 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e560665-fd1d-4093-9418-e2f5ee3809a4 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:21.628656 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e560665-fd1d-4093-9418-e2f5ee3809a4 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:21.629984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 698/1407] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:21.635224 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333831e8-17d9-43f8-ba93-cd493732451a None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:21.638195 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333831e8-17d9-43f8-ba93-cd493732451a None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:21.638507 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333831e8-17d9-43f8-ba93-cd493732451a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:21.710183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-333831e8-17d9-43f8-ba93-cd493732451a None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:21.710183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-333831e8-17d9-43f8-ba93-cd493732451a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:21.815070 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 710/1408] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:22.105301 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e560665-fd1d-4093-9418-e2f5ee3809a4 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:22.106926 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6e560665-fd1d-4093-9418-e2f5ee3809a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:22.110076 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:22.270778 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e560665-fd1d-4093-9418-e2f5ee3809a4 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:22.354540 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:22.403208 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ebdfa-c5a5-41a7-a932-948a9e42bc75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:22.403827 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ebdfa-c5a5-41a7-a932-948a9e42bc75 service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:22.404111 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ebdfa-c5a5-41a7-a932-948a9e42bc75 service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:22.404383 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ebdfa-c5a5-41a7-a932-948a9e42bc75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:22.406052 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-676ebdfa-c5a5-41a7-a932-948a9e42bc75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:22.508688 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ebdfa-c5a5-41a7-a932-948a9e42bc75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:22.510554 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ebdfa-c5a5-41a7-a932-948a9e42bc75 service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:22.533507 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 699/1409] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:27:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:27:31.811014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=67 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:32.530959 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=61 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:41.709117 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205dbbb-4621-4bed-ab5a-d1cb3cec1918 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:41.709646 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205dbbb-4621-4bed-ab5a-d1cb3cec1918 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:41.709646 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205dbbb-4621-4bed-ab5a-d1cb3cec1918 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:41.710975 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 711/1410] 10.222.0.18 () {54 vars in 918 bytes} [Thu Sep 1 16:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:41.716143 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e9569-2f93-413e-afd4-d0591940ed01 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:41.716595 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e9569-2f93-413e-afd4-d0591940ed01 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:41.716698 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e9569-2f93-413e-afd4-d0591940ed01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:41.799832 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-846e9569-2f93-413e-afd4-d0591940ed01 None None] MFA Rules not processed for user `3ddfb151bbf844efae73ec1990e7eaa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:41.801060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-846e9569-2f93-413e-afd4-d0591940ed01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:41.905676 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 700/1411] 10.222.0.18 () {58 vars in 1015 bytes} [Thu Sep 1 16:27:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:41.920481 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94dcc81-5bb6-4c87-8791-73a3edbdcc84 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:41.920768 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94dcc81-5bb6-4c87-8791-73a3edbdcc84 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:41.921048 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94dcc81-5bb6-4c87-8791-73a3edbdcc84 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:41.922412 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 712/1412] 10.222.0.18 () {54 vars in 918 bytes} [Thu Sep 1 16:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:41.926925 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08f128-1dff-4ed9-bc90-3c1561309077 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:41.927215 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08f128-1dff-4ed9-bc90-3c1561309077 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:41.927494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08f128-1dff-4ed9-bc90-3c1561309077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:41.999604 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:42.003757 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-3d08f128-1dff-4ed9-bc90-3c1561309077 None None] MFA Rules not processed for user `278e6d6aed414c758fe3e9f4df6bf81e`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:42.005517 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3d08f128-1dff-4ed9-bc90-3c1561309077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:42.120524 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 701/1413] 10.222.0.18 () {58 vars in 1015 bytes} [Thu Sep 1 16:27:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:44.477156 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfeec24-f914-46bf-81d1-0be9ba2bc5e8 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:44.477729 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfeec24-f914-46bf-81d1-0be9ba2bc5e8 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:44.477729 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfeec24-f914-46bf-81d1-0be9ba2bc5e8 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:44.479115 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 713/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:27:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:44.484620 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf0d70-05ff-45e5-b9d0-17369f5015a0 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:44.485016 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf0d70-05ff-45e5-b9d0-17369f5015a0 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:44.485175 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf0d70-05ff-45e5-b9d0-17369f5015a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:44.553604 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-c5cf0d70-05ff-45e5-b9d0-17369f5015a0 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:44.554848 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c5cf0d70-05ff-45e5-b9d0-17369f5015a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:44.612346 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:44.654752 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 702/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:27:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:44.957122 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfeec24-f914-46bf-81d1-0be9ba2bc5e8 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:44.960032 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-9bfeec24-f914-46bf-81d1-0be9ba2bc5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:45.104637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfeec24-f914-46bf-81d1-0be9ba2bc5e8 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:45.175641 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:45.216362 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3f79b-ef72-4949-9f4e-4aa3d2c67bcb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:45.216902 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3f79b-ef72-4949-9f4e-4aa3d2c67bcb service nova] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:45.217187 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3f79b-ef72-4949-9f4e-4aa3d2c67bcb service nova] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:45.217463 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3f79b-ef72-4949-9f4e-4aa3d2c67bcb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:45.219085 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-2da3f79b-ef72-4949-9f4e-4aa3d2c67bcb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:45.306872 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3f79b-ef72-4949-9f4e-4aa3d2c67bcb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:45.308499 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3f79b-ef72-4949-9f4e-4aa3d2c67bcb service nova] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:45.326678 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 714/1416] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:27:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:27:46.205641 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e26c86-9ce1-43e2-8894-92ec89c5c0af None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:46.206219 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e26c86-9ce1-43e2-8894-92ec89c5c0af None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:46.206219 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e26c86-9ce1-43e2-8894-92ec89c5c0af None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:46.207553 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 703/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:46.212539 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22868ff8-8e7d-4897-8275-0d4b051c2044 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:46.213332 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22868ff8-8e7d-4897-8275-0d4b051c2044 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:46.213332 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22868ff8-8e7d-4897-8275-0d4b051c2044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:46.289615 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-22868ff8-8e7d-4897-8275-0d4b051c2044 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:46.291356 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-22868ff8-8e7d-4897-8275-0d4b051c2044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:46.318001 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:46.387061 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 715/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:27:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:46.535386 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e26c86-9ce1-43e2-8894-92ec89c5c0af None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:46.536909 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-08e26c86-9ce1-43e2-8894-92ec89c5c0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:46.681747 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e26c86-9ce1-43e2-8894-92ec89c5c0af None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:46.721542 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:46.794750 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c858d33-f6a6-4f3e-a780-198852f0de85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:46.795362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c858d33-f6a6-4f3e-a780-198852f0de85 service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:46.795647 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c858d33-f6a6-4f3e-a780-198852f0de85 service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:46.795923 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c858d33-f6a6-4f3e-a780-198852f0de85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:46.797521 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9c858d33-f6a6-4f3e-a780-198852f0de85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:46.886659 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c858d33-f6a6-4f3e-a780-198852f0de85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:46.888249 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c858d33-f6a6-4f3e-a780-198852f0de85 service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:46.906723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 704/1419] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:27:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:27:47.776662 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee72d28-b2e9-4d36-8404-e0f17094a98a None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:47.777201 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee72d28-b2e9-4d36-8404-e0f17094a98a None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:47.777201 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee72d28-b2e9-4d36-8404-e0f17094a98a None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:47.778509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 716/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:27:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:47.783558 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a7e4c-7e7d-424b-b91c-7626c6f5ea27 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:47.783966 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a7e4c-7e7d-424b-b91c-7626c6f5ea27 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:47.784110 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a7e4c-7e7d-424b-b91c-7626c6f5ea27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:47.838627 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:47.862187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-895a7e4c-7e7d-424b-b91c-7626c6f5ea27 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:47.863470 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-895a7e4c-7e7d-424b-b91c-7626c6f5ea27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:47.957949 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 705/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:27:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:48.112777 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee72d28-b2e9-4d36-8404-e0f17094a98a None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:48.114466 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-7ee72d28-b2e9-4d36-8404-e0f17094a98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:48.256626 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee72d28-b2e9-4d36-8404-e0f17094a98a None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:48.268877 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:48.369265 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd59f18-5d52-4d99-b70c-d4bc35d17a21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:48.369689 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd59f18-5d52-4d99-b70c-d4bc35d17a21 service nova] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:48.369970 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd59f18-5d52-4d99-b70c-d4bc35d17a21 service nova] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:48.370262 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd59f18-5d52-4d99-b70c-d4bc35d17a21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:48.371875 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-3fd59f18-5d52-4d99-b70c-d4bc35d17a21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:48.459610 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd59f18-5d52-4d99-b70c-d4bc35d17a21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:48.461219 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd59f18-5d52-4d99-b70c-d4bc35d17a21 service nova] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:48.479182 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 717/1422] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:27:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:27:49.337622 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebc720-1c92-46ac-a8e7-b935159eb503 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:49.338212 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebc720-1c92-46ac-a8e7-b935159eb503 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:49.338212 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebc720-1c92-46ac-a8e7-b935159eb503 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:49.339536 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 706/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:27:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:49.344546 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a197f-acfb-4aa5-8b13-d393bba57846 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:49.344958 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a197f-acfb-4aa5-8b13-d393bba57846 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:49.345096 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a197f-acfb-4aa5-8b13-d393bba57846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:49.365720 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:49.540021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-1f4a197f-acfb-4aa5-8b13-d393bba57846 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:49.541798 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-1f4a197f-acfb-4aa5-8b13-d393bba57846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:49.635146 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 718/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:27:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:49.787743 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebc720-1c92-46ac-a8e7-b935159eb503 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:49.789276 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a3ebc720-1c92-46ac-a8e7-b935159eb503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:49.907028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:49.942352 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebc720-1c92-46ac-a8e7-b935159eb503 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:50.061740 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd8165-bb10-4b4f-827f-e062f237f89f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:50.062376 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd8165-bb10-4b4f-827f-e062f237f89f service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:50.062659 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd8165-bb10-4b4f-827f-e062f237f89f service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:50.062934 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd8165-bb10-4b4f-827f-e062f237f89f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:50.064541 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2cfd8165-bb10-4b4f-827f-e062f237f89f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:50.140658 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:50.155416 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd8165-bb10-4b4f-827f-e062f237f89f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:50.157008 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd8165-bb10-4b4f-827f-e062f237f89f service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:50.174423 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 707/1425] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:27:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:27:51.029994 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cad4b8-2643-4088-af1c-0b1c8629fc1c None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:51.030600 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cad4b8-2643-4088-af1c-0b1c8629fc1c None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:51.030600 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cad4b8-2643-4088-af1c-0b1c8629fc1c None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:51.031914 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 719/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:51.036838 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48174f46-9015-4f03-9d8e-4cc6c21d20f1 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:51.037268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48174f46-9015-4f03-9d8e-4cc6c21d20f1 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:51.037467 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48174f46-9015-4f03-9d8e-4cc6c21d20f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:51.112860 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-48174f46-9015-4f03-9d8e-4cc6c21d20f1 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:51.114059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-48174f46-9015-4f03-9d8e-4cc6c21d20f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:51.206500 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 708/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:51.356483 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cad4b8-2643-4088-af1c-0b1c8629fc1c None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:51.358223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-60cad4b8-2643-4088-af1c-0b1c8629fc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:51.440245 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:51.505484 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cad4b8-2643-4088-af1c-0b1c8629fc1c None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:51.616037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c364f6-01e4-4a16-982e-1b9a4d0ef20d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:51.616557 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c364f6-01e4-4a16-982e-1b9a4d0ef20d service nova] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:51.616834 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c364f6-01e4-4a16-982e-1b9a4d0ef20d service nova] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:51.617104 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c364f6-01e4-4a16-982e-1b9a4d0ef20d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:51.618714 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-e4c364f6-01e4-4a16-982e-1b9a4d0ef20d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:51.670312 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:51.710950 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c364f6-01e4-4a16-982e-1b9a4d0ef20d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:51.713204 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c364f6-01e4-4a16-982e-1b9a4d0ef20d service nova] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:51.730463 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 720/1428] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:27:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:27:52.533406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f69291-10f4-4aac-8cd2-793f144bc6e8 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:52.533978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f69291-10f4-4aac-8cd2-793f144bc6e8 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:52.533978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f69291-10f4-4aac-8cd2-793f144bc6e8 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:52.535314 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 709/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:52.540487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a54508-c736-4c22-b8e2-33a78091f985 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:52.541456 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a54508-c736-4c22-b8e2-33a78091f985 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:52.541691 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a54508-c736-4c22-b8e2-33a78091f985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:52.617374 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-47a54508-c736-4c22-b8e2-33a78091f985 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:52.618679 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-47a54508-c736-4c22-b8e2-33a78091f985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:52.711709 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 721/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:52.863640 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f69291-10f4-4aac-8cd2-793f144bc6e8 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:52.865147 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-12f69291-10f4-4aac-8cd2-793f144bc6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:52.914895 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:53.011440 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f69291-10f4-4aac-8cd2-793f144bc6e8 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:53.123237 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac33c20-9179-48ca-ae59-d4ad7da649ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:53.123844 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac33c20-9179-48ca-ae59-d4ad7da649ff service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:53.124126 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac33c20-9179-48ca-ae59-d4ad7da649ff service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:53.124399 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac33c20-9179-48ca-ae59-d4ad7da649ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:53.125989 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-4ac33c20-9179-48ca-ae59-d4ad7da649ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:53.132791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:53.215572 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac33c20-9179-48ca-ae59-d4ad7da649ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:53.217081 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac33c20-9179-48ca-ae59-d4ad7da649ff service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:53.234739 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 710/1431] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:27:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:27:54.038209 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49396946-c1a3-4356-a1ab-2bdc9a15b1d3 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:54.038753 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49396946-c1a3-4356-a1ab-2bdc9a15b1d3 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:54.038877 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49396946-c1a3-4356-a1ab-2bdc9a15b1d3 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:54.040139 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 722/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:54.045622 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020727ed-d809-4f3f-978d-22487a18fbd0 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:54.046050 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020727ed-d809-4f3f-978d-22487a18fbd0 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:54.046176 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020727ed-d809-4f3f-978d-22487a18fbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:54.121099 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-020727ed-d809-4f3f-978d-22487a18fbd0 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:54.122336 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-020727ed-d809-4f3f-978d-22487a18fbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:54.187910 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:54.217644 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 711/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:54.380314 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49396946-c1a3-4356-a1ab-2bdc9a15b1d3 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:54.381810 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-49396946-c1a3-4356-a1ab-2bdc9a15b1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:54.399745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:54.526068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49396946-c1a3-4356-a1ab-2bdc9a15b1d3 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:54.612450 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:54.639463 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9784f519-3d70-4e6f-b067-27bd22c7e3c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:54.640065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784f519-3d70-4e6f-b067-27bd22c7e3c1 service nova] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:54.640351 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784f519-3d70-4e6f-b067-27bd22c7e3c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:54.640655 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784f519-3d70-4e6f-b067-27bd22c7e3c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:54.642317 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-9784f519-3d70-4e6f-b067-27bd22c7e3c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:54.729704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9784f519-3d70-4e6f-b067-27bd22c7e3c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:54.731332 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9784f519-3d70-4e6f-b067-27bd22c7e3c1 service nova] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:54.749603 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 723/1434] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:27:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:27:55.544236 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdab0cf-f322-4b52-a4b3-6189dbacbe81 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:55.544798 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdab0cf-f322-4b52-a4b3-6189dbacbe81 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:55.544798 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdab0cf-f322-4b52-a4b3-6189dbacbe81 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:55.546158 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 712/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:55.551062 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62090c8-2359-464c-a780-c37511a424e1 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:55.551451 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62090c8-2359-464c-a780-c37511a424e1 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:55.551612 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62090c8-2359-464c-a780-c37511a424e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:55.626725 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-f62090c8-2359-464c-a780-c37511a424e1 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:55.627820 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-f62090c8-2359-464c-a780-c37511a424e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:55.668574 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:55.724931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 724/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:55.877356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdab0cf-f322-4b52-a4b3-6189dbacbe81 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:55.878817 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-cbdab0cf-f322-4b52-a4b3-6189dbacbe81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:56.024197 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdab0cf-f322-4b52-a4b3-6189dbacbe81 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:56.078337 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:56.139377 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b3a71c-7a42-4ff7-8987-5b5d40e8fda8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:56.139925 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3a71c-7a42-4ff7-8987-5b5d40e8fda8 service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:56.140201 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3a71c-7a42-4ff7-8987-5b5d40e8fda8 service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:56.140470 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3a71c-7a42-4ff7-8987-5b5d40e8fda8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:56.142041 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-16b3a71c-7a42-4ff7-8987-5b5d40e8fda8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:56.229675 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b3a71c-7a42-4ff7-8987-5b5d40e8fda8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:56.231269 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b3a71c-7a42-4ff7-8987-5b5d40e8fda8 service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:56.249232 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 713/1437] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:27:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:27:57.070144 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243da20a-eb40-4503-b04e-40615b23370c None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:57.070712 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243da20a-eb40-4503-b04e-40615b23370c None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:57.070815 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243da20a-eb40-4503-b04e-40615b23370c None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:57.072744 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 725/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:27:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:57.077783 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534e2d5-9a68-4a8f-9d7c-bca2d4754a38 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:57.078221 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534e2d5-9a68-4a8f-9d7c-bca2d4754a38 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:57.078365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534e2d5-9a68-4a8f-9d7c-bca2d4754a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:57.152879 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-f534e2d5-9a68-4a8f-9d7c-bca2d4754a38 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:57.154095 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f534e2d5-9a68-4a8f-9d7c-bca2d4754a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:57.156142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:57.250982 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 714/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:27:57.404081 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243da20a-eb40-4503-b04e-40615b23370c None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:57.405649 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-243da20a-eb40-4503-b04e-40615b23370c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:57.548155 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243da20a-eb40-4503-b04e-40615b23370c None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:57.574563 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:57.659927 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f57918-596a-4df0-81e5-84589f627f64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:57.660504 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f57918-596a-4df0-81e5-84589f627f64 service nova] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:57.660792 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f57918-596a-4df0-81e5-84589f627f64 service nova] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:57.661068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f57918-596a-4df0-81e5-84589f627f64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:57.662779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-23f57918-596a-4df0-81e5-84589f627f64 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:57.753175 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f57918-596a-4df0-81e5-84589f627f64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:57.754830 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f57918-596a-4df0-81e5-84589f627f64 service nova] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:57.772179 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 726/1440] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:27:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:27:59.587278 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a2261-186a-457c-9f52-6ecad837fdff None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:59.587842 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a2261-186a-457c-9f52-6ecad837fdff None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:59.587842 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a2261-186a-457c-9f52-6ecad837fdff None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:59.589163 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 715/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:27:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:59.594601 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fbbde-9879-490e-b612-e674a909745b None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:59.595015 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fbbde-9879-490e-b612-e674a909745b None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:59.595156 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fbbde-9879-490e-b612-e674a909745b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:59.637909 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:59.662652 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-160fbbde-9879-490e-b612-e674a909745b None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:59.663859 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-160fbbde-9879-490e-b612-e674a909745b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:59.759993 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 727/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:27:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:28:00.046742 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a2261-186a-457c-9f52-6ecad837fdff None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:00.048282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-360a2261-186a-457c-9f52-6ecad837fdff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:00.151719 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a2261-186a-457c-9f52-6ecad837fdff None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:00.183501 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:00.224238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226d053-5aa8-42ab-b22e-c7f41109bbec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:00.224850 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226d053-5aa8-42ab-b22e-c7f41109bbec service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:00.225131 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226d053-5aa8-42ab-b22e-c7f41109bbec service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:00.225410 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226d053-5aa8-42ab-b22e-c7f41109bbec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:00.227023 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3226d053-5aa8-42ab-b22e-c7f41109bbec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:00.452332 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3226d053-5aa8-42ab-b22e-c7f41109bbec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:00.453787 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3226d053-5aa8-42ab-b22e-c7f41109bbec service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:00.471577 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 716/1443] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:28:02.513854 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4a97c-8baa-4b9e-8686-b9eecd0f2e8d None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:02.514441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4a97c-8baa-4b9e-8686-b9eecd0f2e8d None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:02.514441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4a97c-8baa-4b9e-8686-b9eecd0f2e8d None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:02.515781 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 728/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:02.521128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272721a-e7a5-46e8-ad5b-8b783353d278 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:02.521547 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272721a-e7a5-46e8-ad5b-8b783353d278 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:02.521667 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272721a-e7a5-46e8-ad5b-8b783353d278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:02.585580 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-6272721a-e7a5-46e8-ad5b-8b783353d278 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:02.586865 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6272721a-e7a5-46e8-ad5b-8b783353d278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:02.599335 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:02.682913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 717/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:28:02.966753 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d4a97c-8baa-4b9e-8686-b9eecd0f2e8d None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:02.968280 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-d7d4a97c-8baa-4b9e-8686-b9eecd0f2e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:03.074685 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d4a97c-8baa-4b9e-8686-b9eecd0f2e8d None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:03.086456 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:03.146266 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe6d64e-58ab-4e2a-af79-bd271bfa4190 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:03.146857 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6d64e-58ab-4e2a-af79-bd271bfa4190 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:03.147139 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6d64e-58ab-4e2a-af79-bd271bfa4190 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:03.147414 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6d64e-58ab-4e2a-af79-bd271bfa4190 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:03.149002 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-bfe6d64e-58ab-4e2a-af79-bd271bfa4190 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:03.254660 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe6d64e-58ab-4e2a-af79-bd271bfa4190 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:03.256023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe6d64e-58ab-4e2a-af79-bd271bfa4190 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:03.273791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 729/1446] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:28:05.453919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce7260-a16e-4cc0-85f6-d54008c9bd87 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:05.454499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce7260-a16e-4cc0-85f6-d54008c9bd87 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:05.454499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce7260-a16e-4cc0-85f6-d54008c9bd87 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:05.455829 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 718/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:05.461098 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af832f91-3bae-4ff1-9d95-3f81f77ce0f2 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:05.461525 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af832f91-3bae-4ff1-9d95-3f81f77ce0f2 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:05.461656 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af832f91-3bae-4ff1-9d95-3f81f77ce0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:05.510712 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:05.529004 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-af832f91-3bae-4ff1-9d95-3f81f77ce0f2 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:05.530198 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-af832f91-3bae-4ff1-9d95-3f81f77ce0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:05.623390 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 730/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:28:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:05.906944 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce7260-a16e-4cc0-85f6-d54008c9bd87 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:05.908477 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-4cce7260-a16e-4cc0-85f6-d54008c9bd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:06.010089 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce7260-a16e-4cc0-85f6-d54008c9bd87 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:06.050123 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:06.083230 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b691d-68aa-42a8-b387-9426b143bd5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:06.083832 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b691d-68aa-42a8-b387-9426b143bd5a service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:06.084110 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b691d-68aa-42a8-b387-9426b143bd5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:06.084380 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b691d-68aa-42a8-b387-9426b143bd5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:06.085953 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-192b691d-68aa-42a8-b387-9426b143bd5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:06.194281 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b691d-68aa-42a8-b387-9426b143bd5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:06.195641 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b691d-68aa-42a8-b387-9426b143bd5a service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:06.213504 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 719/1449] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:28:08.218509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd648211-cff9-41c8-a31a-dea7ed714c60 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:08.219049 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd648211-cff9-41c8-a31a-dea7ed714c60 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:08.219049 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd648211-cff9-41c8-a31a-dea7ed714c60 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:08.220365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 731/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:08.225693 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256a56c-6566-4561-8a2e-ff7da6494cd0 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:08.226108 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256a56c-6566-4561-8a2e-ff7da6494cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:08.226268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256a56c-6566-4561-8a2e-ff7da6494cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:08.291753 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-2256a56c-6566-4561-8a2e-ff7da6494cd0 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:08.292946 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2256a56c-6566-4561-8a2e-ff7da6494cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:08.313393 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:08.389118 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 720/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:08.669794 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd648211-cff9-41c8-a31a-dea7ed714c60 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:08.671355 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-fd648211-cff9-41c8-a31a-dea7ed714c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:08.773062 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd648211-cff9-41c8-a31a-dea7ed714c60 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:08.793403 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:08.846058 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d5fbd-eaa0-4bfa-b4b3-b64f2d5241b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:08.846688 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d5fbd-eaa0-4bfa-b4b3-b64f2d5241b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:08.846997 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d5fbd-eaa0-4bfa-b4b3-b64f2d5241b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:08.847291 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d5fbd-eaa0-4bfa-b4b3-b64f2d5241b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:08.848862 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-261d5fbd-eaa0-4bfa-b4b3-b64f2d5241b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:08.956243 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d5fbd-eaa0-4bfa-b4b3-b64f2d5241b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:08.957620 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d5fbd-eaa0-4bfa-b4b3-b64f2d5241b0 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:08.976018 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 732/1452] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:28:11.046332 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499db8e-269f-4662-8b9c-0f5d95d7d62d None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:11.046905 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499db8e-269f-4662-8b9c-0f5d95d7d62d None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:11.046905 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499db8e-269f-4662-8b9c-0f5d95d7d62d None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:11.048235 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 721/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:28:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:11.053527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2cf2a-439c-41af-9d4a-7d82917081de None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:11.053954 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2cf2a-439c-41af-9d4a-7d82917081de None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:11.054064 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2cf2a-439c-41af-9d4a-7d82917081de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:11.116853 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:11.120955 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-60e2cf2a-439c-41af-9d4a-7d82917081de None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:11.122316 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-60e2cf2a-439c-41af-9d4a-7d82917081de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:11.216391 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 733/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:11.506983 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a499db8e-269f-4662-8b9c-0f5d95d7d62d None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:11.508511 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a499db8e-269f-4662-8b9c-0f5d95d7d62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:11.611635 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a499db8e-269f-4662-8b9c-0f5d95d7d62d None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:11.659810 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:11.685524 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba868f53-a632-4592-8e3b-c611167b3814 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:11.686129 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba868f53-a632-4592-8e3b-c611167b3814 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:11.686431 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba868f53-a632-4592-8e3b-c611167b3814 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:11.686705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba868f53-a632-4592-8e3b-c611167b3814 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:11.688285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ba868f53-a632-4592-8e3b-c611167b3814 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:11.795126 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba868f53-a632-4592-8e3b-c611167b3814 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:11.796652 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba868f53-a632-4592-8e3b-c611167b3814 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:11.814339 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 722/1455] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:28:14.002675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7766c52-ae04-4bff-a121-dff9d69df028 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:14.003259 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7766c52-ae04-4bff-a121-dff9d69df028 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:14.003259 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7766c52-ae04-4bff-a121-dff9d69df028 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:14.004557 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 734/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:28:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:14.010341 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b36e6-c675-4e72-a660-fa922214397c None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:14.010756 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b36e6-c675-4e72-a660-fa922214397c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:14.010897 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b36e6-c675-4e72-a660-fa922214397c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:14.077628 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-eb7b36e6-c675-4e72-a660-fa922214397c None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:14.078850 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-eb7b36e6-c675-4e72-a660-fa922214397c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:14.106362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:14.173300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 723/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:28:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:14.456870 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7766c52-ae04-4bff-a121-dff9d69df028 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:14.458473 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-c7766c52-ae04-4bff-a121-dff9d69df028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:14.561647 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7766c52-ae04-4bff-a121-dff9d69df028 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:14.588794 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:14.633537 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633106d0-c11b-4ac2-9ac2-e6ce3420c94b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:14.634067 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633106d0-c11b-4ac2-9ac2-e6ce3420c94b service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:14.634532 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633106d0-c11b-4ac2-9ac2-e6ce3420c94b service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:14.634809 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633106d0-c11b-4ac2-9ac2-e6ce3420c94b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:14.636384 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-633106d0-c11b-4ac2-9ac2-e6ce3420c94b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:14.743048 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633106d0-c11b-4ac2-9ac2-e6ce3420c94b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:14.744498 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633106d0-c11b-4ac2-9ac2-e6ce3420c94b service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:14.762532 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 735/1458] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:28:16.717408 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406b019-8573-4066-a518-2adccc19ee6f None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:16.717408 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406b019-8573-4066-a518-2adccc19ee6f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:16.717408 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406b019-8573-4066-a518-2adccc19ee6f None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:16.717408 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 724/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:28:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:16.720990 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0e2f3-a29b-4162-8b84-bc28b11e68fc None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:16.721374 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0e2f3-a29b-4162-8b84-bc28b11e68fc None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:16.721538 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0e2f3-a29b-4162-8b84-bc28b11e68fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:16.786391 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-90a0e2f3-a29b-4162-8b84-bc28b11e68fc None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:16.787600 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-90a0e2f3-a29b-4162-8b84-bc28b11e68fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:16.795711 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:16.882773 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 736/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:28:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:17.166456 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406b019-8573-4066-a518-2adccc19ee6f None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:17.167990 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0406b019-8573-4066-a518-2adccc19ee6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:17.270914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406b019-8573-4066-a518-2adccc19ee6f None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:17.325373 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:17.343430 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a162f-8005-4f13-b007-e9c41a391052 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:17.344033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a162f-8005-4f13-b007-e9c41a391052 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:17.344315 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a162f-8005-4f13-b007-e9c41a391052 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:17.344590 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a162f-8005-4f13-b007-e9c41a391052 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:17.346168 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b44a162f-8005-4f13-b007-e9c41a391052 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:17.454277 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a162f-8005-4f13-b007-e9c41a391052 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:17.455651 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a162f-8005-4f13-b007-e9c41a391052 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:17.473599 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 725/1461] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:28:19.427443 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1aece-c12b-4cd0-8c50-853941e05279 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:19.427933 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1aece-c12b-4cd0-8c50-853941e05279 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:19.427933 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1aece-c12b-4cd0-8c50-853941e05279 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:19.429275 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 737/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:19.434606 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46afa4-e500-420c-845d-4d6f83d809af None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:19.435048 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46afa4-e500-420c-845d-4d6f83d809af None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:19.435169 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46afa4-e500-420c-845d-4d6f83d809af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:19.501177 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-3e46afa4-e500-420c-845d-4d6f83d809af None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:19.502433 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3e46afa4-e500-420c-845d-4d6f83d809af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:19.540768 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:19.599839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 726/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:28:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:19.894044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1aece-c12b-4cd0-8c50-853941e05279 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:19.895645 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-fbc1aece-c12b-4cd0-8c50-853941e05279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:20.000465 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1aece-c12b-4cd0-8c50-853941e05279 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:20.037631 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:20.073627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38859f84-d5ce-4a07-87e9-db72f8789b26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:20.074246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38859f84-d5ce-4a07-87e9-db72f8789b26 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:20.074545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38859f84-d5ce-4a07-87e9-db72f8789b26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:20.074819 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38859f84-d5ce-4a07-87e9-db72f8789b26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:20.076411 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-38859f84-d5ce-4a07-87e9-db72f8789b26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:20.183426 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38859f84-d5ce-4a07-87e9-db72f8789b26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:20.184803 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38859f84-d5ce-4a07-87e9-db72f8789b26 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:20.203234 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 738/1464] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:28:22.098407 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434098ed-bf1e-4ec4-9f25-ec7c76f0474d None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:22.098972 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434098ed-bf1e-4ec4-9f25-ec7c76f0474d None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:22.098972 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434098ed-bf1e-4ec4-9f25-ec7c76f0474d None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:22.100282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 727/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:22.106084 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722bc52-3f97-401d-a8f4-0404bfd53ff5 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:22.106503 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722bc52-3f97-401d-a8f4-0404bfd53ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:22.106679 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722bc52-3f97-401d-a8f4-0404bfd53ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:22.172315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-b722bc52-3f97-401d-a8f4-0404bfd53ff5 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:22.173516 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-b722bc52-3f97-401d-a8f4-0404bfd53ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:22.191519 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:22.269813 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 739/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:28:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:22.552575 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434098ed-bf1e-4ec4-9f25-ec7c76f0474d None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:22.554127 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-434098ed-bf1e-4ec4-9f25-ec7c76f0474d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:22.656214 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434098ed-bf1e-4ec4-9f25-ec7c76f0474d None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:22.721057 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:22.727981 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1dfb73-ea5a-4fbd-b07e-cd0dff3961a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:22.728612 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1dfb73-ea5a-4fbd-b07e-cd0dff3961a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:22.728897 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1dfb73-ea5a-4fbd-b07e-cd0dff3961a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:22.729170 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1dfb73-ea5a-4fbd-b07e-cd0dff3961a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:22.730781 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-dc1dfb73-ea5a-4fbd-b07e-cd0dff3961a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:22.837249 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1dfb73-ea5a-4fbd-b07e-cd0dff3961a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:22.839338 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1dfb73-ea5a-4fbd-b07e-cd0dff3961a9 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:22.856764 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 728/1467] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:28:24.758367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850e234-39b8-46be-b052-443b3766ce90 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:24.758972 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850e234-39b8-46be-b052-443b3766ce90 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:24.758972 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850e234-39b8-46be-b052-443b3766ce90 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:24.760296 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 740/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:28:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:24.765597 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1caae-d638-4f98-99ab-2dbb09333aac None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:24.766007 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1caae-d638-4f98-99ab-2dbb09333aac None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:24.766165 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1caae-d638-4f98-99ab-2dbb09333aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:24.832170 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-21b1caae-d638-4f98-99ab-2dbb09333aac None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:24.833319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-21b1caae-d638-4f98-99ab-2dbb09333aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:24.880701 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:24.930581 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 729/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:28:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:25.212771 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850e234-39b8-46be-b052-443b3766ce90 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:25.214326 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-6850e234-39b8-46be-b052-443b3766ce90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:25.314526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850e234-39b8-46be-b052-443b3766ce90 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:25.361519 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:25.388506 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae461fa9-1c06-44a5-b88b-7ee3f6bddf27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:25.389117 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae461fa9-1c06-44a5-b88b-7ee3f6bddf27 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:25.389400 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae461fa9-1c06-44a5-b88b-7ee3f6bddf27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:25.389674 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae461fa9-1c06-44a5-b88b-7ee3f6bddf27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:25.391294 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-ae461fa9-1c06-44a5-b88b-7ee3f6bddf27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:25.498347 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae461fa9-1c06-44a5-b88b-7ee3f6bddf27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:25.499664 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae461fa9-1c06-44a5-b88b-7ee3f6bddf27 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:25.518793 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 741/1470] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:28:34.928808 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=22 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:35.516486 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=66 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:45.031042 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32287cde-7861-4867-92b2-24ebed5914d1 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:45.031613 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32287cde-7861-4867-92b2-24ebed5914d1 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:45.031613 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32287cde-7861-4867-92b2-24ebed5914d1 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:45.032981 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 730/1471] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:28:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:45.038455 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7924dd-2979-4b85-ad99-e8eccb9ce9cd None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:45.038899 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7924dd-2979-4b85-ad99-e8eccb9ce9cd None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:45.039004 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7924dd-2979-4b85-ad99-e8eccb9ce9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:45.109085 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-6f7924dd-2979-4b85-ad99-e8eccb9ce9cd None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:45.110324 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-6f7924dd-2979-4b85-ad99-e8eccb9ce9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:45.205813 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 742/1472] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:28:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:45.508134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b542f2-ea5a-4f3f-8db4-2157317aa8a2 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:45.508439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b542f2-ea5a-4f3f-8db4-2157317aa8a2 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:45.508742 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b542f2-ea5a-4f3f-8db4-2157317aa8a2 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:45.510814 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 731/1473] 10.222.0.18 () {54 vars in 967 bytes} [Thu Sep 1 16:28:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:45.516103 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c491620-c989-4987-9955-2ed481a42875 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:45.516396 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c491620-c989-4987-9955-2ed481a42875 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:45.516674 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c491620-c989-4987-9955-2ed481a42875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:45.593472 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-7c491620-c989-4987-9955-2ed481a42875 None None] MFA Rules not processed for user `b09f6d550c9142bf956b225bf5172e42`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:45.594762 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-7c491620-c989-4987-9955-2ed481a42875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:45.615807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:45.671215 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 743/1474] 10.222.0.18 () {58 vars in 1064 bytes} [Thu Sep 1 16:28:45 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:45.678718 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b542f2-ea5a-4f3f-8db4-2157317aa8a2 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:45.680281 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-26b542f2-ea5a-4f3f-8db4-2157317aa8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:45.788293 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b542f2-ea5a-4f3f-8db4-2157317aa8a2 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:45.863559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b4658-9821-4234-99cd-da88c9f94867 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b09f6d550c9142bf956b225bf5172e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:45.864173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b4658-9821-4234-99cd-da88c9f94867 service glance] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:45.864456 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b4658-9821-4234-99cd-da88c9f94867 service glance] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:45.864732 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b4658-9821-4234-99cd-da88c9f94867 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:45.866366 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6e0b4658-9821-4234-99cd-da88c9f94867 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:45.922045 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:45.976073 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b4658-9821-4234-99cd-da88c9f94867 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:45.977502 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b4658-9821-4234-99cd-da88c9f94867 service glance] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:45.996189 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 732/1475] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:28:47.988438 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bb74e-9765-4d72-92f4-0d5d888f0554 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:47.989046 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bb74e-9765-4d72-92f4-0d5d888f0554 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:47.989046 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bb74e-9765-4d72-92f4-0d5d888f0554 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:47.990448 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 744/1476] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:28:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:47.995487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9484f-d239-44df-880d-21881cf9fd61 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:47.995880 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9484f-d239-44df-880d-21881cf9fd61 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:47.996028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9484f-d239-44df-880d-21881cf9fd61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:48.061630 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-aeb9484f-d239-44df-880d-21881cf9fd61 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:48.062857 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-aeb9484f-d239-44df-880d-21881cf9fd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:48.158858 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 733/1477] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:28:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:48.427411 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705bb74e-9765-4d72-92f4-0d5d888f0554 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:48.428937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-705bb74e-9765-4d72-92f4-0d5d888f0554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:48.553882 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705bb74e-9765-4d72-92f4-0d5d888f0554 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:48.605427 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:48.648575 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d19d3-0115-4a47-ad3f-21809e85f59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:48.649110 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d19d3-0115-4a47-ad3f-21809e85f59a admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:48.649385 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d19d3-0115-4a47-ad3f-21809e85f59a admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:48.649654 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d19d3-0115-4a47-ad3f-21809e85f59a admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:48.650847 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d19d3-0115-4a47-ad3f-21809e85f59a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:48.652558 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: WARNING py.warnings [None req-989d19d3-0115-4a47-ad3f-21809e85f59a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:28:48.652558 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: warnings.warn(msg) Sep 01 16:28:48.652558 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: Sep 01 16:28:48.653706 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d19d3-0115-4a47-ad3f-21809e85f59a admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:48.672742 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 745/1478] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Sep 1 16:28:48 2022] GET /identity/v3/projects => generated 2553 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:28:50.597207 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b3863-c434-430b-9816-abe80b1a7f37 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:50.597783 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b3863-c434-430b-9816-abe80b1a7f37 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:50.597783 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b3863-c434-430b-9816-abe80b1a7f37 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:50.599148 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 734/1479] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:28:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:28:50.605022 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c3a5f-ca6a-406d-9425-3046c3796887 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:50.605506 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c3a5f-ca6a-406d-9425-3046c3796887 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:50.605617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c3a5f-ca6a-406d-9425-3046c3796887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:50.671866 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-7d1c3a5f-ca6a-406d-9425-3046c3796887 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:50.673076 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-7d1c3a5f-ca6a-406d-9425-3046c3796887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:50.771942 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 746/1480] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:28:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:51.066319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2b3863-c434-430b-9816-abe80b1a7f37 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:51.067871 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ab2b3863-c434-430b-9816-abe80b1a7f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:51.078329 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:51.219782 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2b3863-c434-430b-9816-abe80b1a7f37 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:51.307799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:51.340787 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f0c6c-0528-46a4-a366-3d511ddf57ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:51.341422 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f0c6c-0528-46a4-a366-3d511ddf57ae service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:51.341703 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f0c6c-0528-46a4-a366-3d511ddf57ae service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:51.341972 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f0c6c-0528-46a4-a366-3d511ddf57ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:51.343596 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-fd7f0c6c-0528-46a4-a366-3d511ddf57ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:51.434896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7f0c6c-0528-46a4-a366-3d511ddf57ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:51.436595 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7f0c6c-0528-46a4-a366-3d511ddf57ae service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:51.454596 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 735/1481] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:28:53.541345 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d14a5-976a-4d00-a633-d477c4ae6feb None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:53.541943 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d14a5-976a-4d00-a633-d477c4ae6feb None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:53.541943 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d14a5-976a-4d00-a633-d477c4ae6feb None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:53.543271 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 747/1482] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:28:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:53.548308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93b6f4-b0b1-4e0c-9528-ffbe1bfa4e00 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:53.548716 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93b6f4-b0b1-4e0c-9528-ffbe1bfa4e00 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:53.548856 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93b6f4-b0b1-4e0c-9528-ffbe1bfa4e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:53.613888 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-8a93b6f4-b0b1-4e0c-9528-ffbe1bfa4e00 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:53.615104 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8a93b6f4-b0b1-4e0c-9528-ffbe1bfa4e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:53.660501 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:53.714011 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 736/1483] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:53.996902 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d14a5-976a-4d00-a633-d477c4ae6feb None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:53.998461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-833d14a5-976a-4d00-a633-d477c4ae6feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:54.021606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:54.101592 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d14a5-976a-4d00-a633-d477c4ae6feb None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:54.175559 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef69fb-4f01-4703-8a17-d841002add3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:54.176169 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef69fb-4f01-4703-8a17-d841002add3d service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:54.176454 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef69fb-4f01-4703-8a17-d841002add3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:54.176777 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef69fb-4f01-4703-8a17-d841002add3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:54.178401 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-08ef69fb-4f01-4703-8a17-d841002add3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:54.253461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:54.289633 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ef69fb-4f01-4703-8a17-d841002add3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:54.291216 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ef69fb-4f01-4703-8a17-d841002add3d service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:54.309693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 748/1484] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:28:56.247881 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed30e4a-0611-4063-a33b-f702c555fdd3 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:56.248494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed30e4a-0611-4063-a33b-f702c555fdd3 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:56.248494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed30e4a-0611-4063-a33b-f702c555fdd3 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:56.249750 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 737/1485] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:28:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:56.256141 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6568d-7e36-4d58-ae26-5ffb116ebdc9 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:56.256562 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6568d-7e36-4d58-ae26-5ffb116ebdc9 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:56.256752 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6568d-7e36-4d58-ae26-5ffb116ebdc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:56.328817 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-3bb6568d-7e36-4d58-ae26-5ffb116ebdc9 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:56.330043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-3bb6568d-7e36-4d58-ae26-5ffb116ebdc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:56.428772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 749/1486] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:28:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:56.715067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed30e4a-0611-4063-a33b-f702c555fdd3 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:56.716598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3ed30e4a-0611-4063-a33b-f702c555fdd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:56.819486 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed30e4a-0611-4063-a33b-f702c555fdd3 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:56.890199 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:56.892152 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37bfda-0946-49fd-ae46-d6b70b42b574 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:56.892778 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37bfda-0946-49fd-ae46-d6b70b42b574 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:56.893256 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37bfda-0946-49fd-ae46-d6b70b42b574 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:56.893922 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37bfda-0946-49fd-ae46-d6b70b42b574 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:56.896094 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9f37bfda-0946-49fd-ae46-d6b70b42b574 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:57.008474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37bfda-0946-49fd-ae46-d6b70b42b574 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:57.010388 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37bfda-0946-49fd-ae46-d6b70b42b574 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:57.029133 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 738/1487] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:28:59.105579 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d6122-e084-4eaa-b49b-718514d205ec None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:59.106163 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d6122-e084-4eaa-b49b-718514d205ec None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:59.106163 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d6122-e084-4eaa-b49b-718514d205ec None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:59.108060 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 750/1488] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:28:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:59.114153 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e10a2-fae7-476e-b8ea-3d43549507eb None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:59.114599 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e10a2-fae7-476e-b8ea-3d43549507eb None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:59.114820 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e10a2-fae7-476e-b8ea-3d43549507eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:59.185648 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-414e10a2-fae7-476e-b8ea-3d43549507eb None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:59.186900 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-414e10a2-fae7-476e-b8ea-3d43549507eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:59.240080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:59.290024 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 739/1489] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:28:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:59.589200 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d6122-e084-4eaa-b49b-718514d205ec None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:59.590745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-de3d6122-e084-4eaa-b49b-718514d205ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:59.627331 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:59.693830 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d6122-e084-4eaa-b49b-718514d205ec None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:59.764814 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1b74a-d1a9-40a8-b11f-58e986344c65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:59.765421 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1b74a-d1a9-40a8-b11f-58e986344c65 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:59.765696 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1b74a-d1a9-40a8-b11f-58e986344c65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:59.765964 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1b74a-d1a9-40a8-b11f-58e986344c65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:59.767573 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-15c1b74a-d1a9-40a8-b11f-58e986344c65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:59.846651 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:59.876347 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1b74a-d1a9-40a8-b11f-58e986344c65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:59.877853 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1b74a-d1a9-40a8-b11f-58e986344c65 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:59.896183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 751/1490] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:29:01.985304 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f33db1-cd8f-4dba-8f68-dc028825d7df None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:01.985871 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f33db1-cd8f-4dba-8f68-dc028825d7df None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:01.985871 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f33db1-cd8f-4dba-8f68-dc028825d7df None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:01.987267 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 740/1491] 10.222.0.18 () {54 vars in 927 bytes} [Thu Sep 1 16:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:29:01.992289 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfd5c5-2b0f-4af1-b5da-772289987a12 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:01.992679 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfd5c5-2b0f-4af1-b5da-772289987a12 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:01.992837 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfd5c5-2b0f-4af1-b5da-772289987a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:02.057131 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-94cfd5c5-2b0f-4af1-b5da-772289987a12 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:02.058354 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-94cfd5c5-2b0f-4af1-b5da-772289987a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:02.154849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 752/1492] 10.222.0.18 () {58 vars in 1024 bytes} [Thu Sep 1 16:29:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:02.446687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f33db1-cd8f-4dba-8f68-dc028825d7df None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:29:02.448805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-02f33db1-cd8f-4dba-8f68-dc028825d7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:02.552564 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f33db1-cd8f-4dba-8f68-dc028825d7df None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:29:02.626211 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b856317-9465-4e72-a2d2-592569f9e2bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:29:02.626766 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b856317-9465-4e72-a2d2-592569f9e2bd service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:02.627060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b856317-9465-4e72-a2d2-592569f9e2bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:02.627321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b856317-9465-4e72-a2d2-592569f9e2bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:02.628904 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-5b856317-9465-4e72-a2d2-592569f9e2bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:02.638392 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:02.739230 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b856317-9465-4e72-a2d2-592569f9e2bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:29:02.740664 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b856317-9465-4e72-a2d2-592569f9e2bd service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:29:02.758906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 741/1493] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:29:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:29:12.152971 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=82 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:12.756505 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=50 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:20.431939 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b64cf39-ba14-46ac-9658-ad0a27d092c9 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:20.432531 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b64cf39-ba14-46ac-9658-ad0a27d092c9 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:20.432531 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b64cf39-ba14-46ac-9658-ad0a27d092c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:20.512541 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-6b64cf39-ba14-46ac-9658-ad0a27d092c9 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:20.513819 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-6b64cf39-ba14-46ac-9658-ad0a27d092c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:20.611090 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 753/1494] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:20.619203 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a79e7-1d87-4ced-8dfd-eeeec99fc199 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:20.619703 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a79e7-1d87-4ced-8dfd-eeeec99fc199 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:20.619817 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a79e7-1d87-4ced-8dfd-eeeec99fc199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:20.696963 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-130a79e7-1d87-4ced-8dfd-eeeec99fc199 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:20.698101 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-130a79e7-1d87-4ced-8dfd-eeeec99fc199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:20.794663 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 742/1495] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:20.799299 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b64cf39-ba14-46ac-9658-ad0a27d092c9 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:20.800514 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-6b64cf39-ba14-46ac-9658-ad0a27d092c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:20.812929 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b64cf39-ba14-46ac-9658-ad0a27d092c9 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:20.874605 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:20.909161 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6bdbf-aa26-4e6c-bba5-bcfd0dd76dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:20.909775 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6bdbf-aa26-4e6c-bba5-bcfd0dd76dfb admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:20.910059 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6bdbf-aa26-4e6c-bba5-bcfd0dd76dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:20.910367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6bdbf-aa26-4e6c-bba5-bcfd0dd76dfb admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:20.911016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb6bdbf-aa26-4e6c-bba5-bcfd0dd76dfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:30:20.911322 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb6bdbf-aa26-4e6c-bba5-bcfd0dd76dfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:20.912339 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: WARNING py.warnings [None req-beb6bdbf-aa26-4e6c-bba5-bcfd0dd76dfb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:30:20.912339 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: warnings.warn(msg) Sep 01 16:30:20.912339 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: Sep 01 16:30:20.913237 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb6bdbf-aa26-4e6c-bba5-bcfd0dd76dfb admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:20.926547 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 754/1496] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:30:20.935599 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a79e7-1d87-4ced-8dfd-eeeec99fc199 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:20.944750 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a79e7-1d87-4ced-8dfd-eeeec99fc199 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:21.005697 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:21.040077 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70f422-e4ab-4662-bf8b-9983955844a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:21.040688 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70f422-e4ab-4662-bf8b-9983955844a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:21.040965 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70f422-e4ab-4662-bf8b-9983955844a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:21.041233 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70f422-e4ab-4662-bf8b-9983955844a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:21.041861 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc70f422-e4ab-4662-bf8b-9983955844a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:21.044265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: WARNING py.warnings [None req-dc70f422-e4ab-4662-bf8b-9983955844a0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:30:21.044265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: warnings.warn(msg) Sep 01 16:30:21.044265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: Sep 01 16:30:21.045454 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc70f422-e4ab-4662-bf8b-9983955844a0 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:21.100848 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 743/1497] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:30:20 2022] POST /identity/v3/projects => generated 379 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:30:21.105201 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6bdbf-aa26-4e6c-bba5-bcfd0dd76dfb admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:21.107372 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-beb6bdbf-aa26-4e6c-bba5-bcfd0dd76dfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:21.140795 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6bdbf-aa26-4e6c-bba5-bcfd0dd76dfb admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:21.237551 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b52b4e-f406-4e47-8355-e2accc941d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:21.238495 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b52b4e-f406-4e47-8355-e2accc941d3a admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:21.238495 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b52b4e-f406-4e47-8355-e2accc941d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:21.238757 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b52b4e-f406-4e47-8355-e2accc941d3a admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:21.239375 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b52b4e-f406-4e47-8355-e2accc941d3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:21.240416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: WARNING py.warnings [None req-05b52b4e-f406-4e47-8355-e2accc941d3a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:30:21.240416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: warnings.warn(msg) Sep 01 16:30:21.240416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: Sep 01 16:30:21.245663 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b52b4e-f406-4e47-8355-e2accc941d3a admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:21.297723 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 755/1498] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:30:21 2022] POST /identity/v3/users => generated 290 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:30:21.301899 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70f422-e4ab-4662-bf8b-9983955844a0 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:21.303459 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-dc70f422-e4ab-4662-bf8b-9983955844a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:21.367314 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70f422-e4ab-4662-bf8b-9983955844a0 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:21.452096 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:21.465526 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d67e0-80f4-4ce3-b398-34636e7e340e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:21.466139 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d67e0-80f4-4ce3-b398-34636e7e340e admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:21.466444 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d67e0-80f4-4ce3-b398-34636e7e340e admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:21.466712 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d67e0-80f4-4ce3-b398-34636e7e340e admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:21.467348 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d67e0-80f4-4ce3-b398-34636e7e340e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:21.468342 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: WARNING py.warnings [None req-785d67e0-80f4-4ce3-b398-34636e7e340e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:30:21.468342 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: warnings.warn(msg) Sep 01 16:30:21.468342 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: Sep 01 16:30:21.469225 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d67e0-80f4-4ce3-b398-34636e7e340e admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:21.482130 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 744/1499] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:30:21.486216 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b52b4e-f406-4e47-8355-e2accc941d3a admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:21.487875 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-05b52b4e-f406-4e47-8355-e2accc941d3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:21.499678 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b52b4e-f406-4e47-8355-e2accc941d3a admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:21.525090 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:21.599350 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a264de80-895e-4fd0-be77-f99b727d4216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:21.599887 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264de80-895e-4fd0-be77-f99b727d4216 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:21.600170 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264de80-895e-4fd0-be77-f99b727d4216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:21.600441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264de80-895e-4fd0-be77-f99b727d4216 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:21.601086 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a264de80-895e-4fd0-be77-f99b727d4216 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:21.602105 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: WARNING py.warnings [None req-a264de80-895e-4fd0-be77-f99b727d4216 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:30:21.602105 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: warnings.warn(msg) Sep 01 16:30:21.602105 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: Sep 01 16:30:21.603031 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a264de80-895e-4fd0-be77-f99b727d4216 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:21.615844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 756/1500] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:30:21.628507 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d67e0-80f4-4ce3-b398-34636e7e340e admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:21.630037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-785d67e0-80f4-4ce3-b398-34636e7e340e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:21.663655 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d67e0-80f4-4ce3-b398-34636e7e340e admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:21.771020 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f307dce-8a17-40cd-aba6-537804314dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:21.771693 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f307dce-8a17-40cd-aba6-537804314dbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:21.771974 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f307dce-8a17-40cd-aba6-537804314dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:21.772273 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f307dce-8a17-40cd-aba6-537804314dbe admin admin] PATH_INFO: `/v3/projects/a5d6662e6cf24b06a9ec0a5159fbfcd7/users/e04386f72f1b432aa6f40c0b3d1b9338/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:21.795262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f307dce-8a17-40cd-aba6-537804314dbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5d6662e6cf24b06a9ec0a5159fbfcd7, user_id=e04386f72f1b432aa6f40c0b3d1b9338, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:21.796292 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: WARNING py.warnings [None req-8f307dce-8a17-40cd-aba6-537804314dbe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:30:21.796292 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: warnings.warn(msg) Sep 01 16:30:21.796292 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: Sep 01 16:30:21.798259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f307dce-8a17-40cd-aba6-537804314dbe admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:21.827255 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 745/1501] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:30:21 2022] PUT /identity/v3/projects/a5d6662e6cf24b06a9ec0a5159fbfcd7/users/e04386f72f1b432aa6f40c0b3d1b9338/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:30:21.839364 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4006f6-f55e-4649-a288-bfdf10915c9c None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:21.839680 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4006f6-f55e-4649-a288-bfdf10915c9c None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:21.839960 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4006f6-f55e-4649-a288-bfdf10915c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:21.900019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-fe4006f6-f55e-4649-a288-bfdf10915c9c None None] MFA Rules not processed for user `e04386f72f1b432aa6f40c0b3d1b9338`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:21.901173 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-fe4006f6-f55e-4649-a288-bfdf10915c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:21.977967 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 757/1502] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:30:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:21.991688 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace39f14-9a89-4d25-9274-e2dea5dd7c46 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:21.991975 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace39f14-9a89-4d25-9274-e2dea5dd7c46 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:21.992251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace39f14-9a89-4d25-9274-e2dea5dd7c46 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:21.993644 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 746/1503] 10.222.0.18 () {54 vars in 964 bytes} [Thu Sep 1 16:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:30:21.999151 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e2485-0361-4299-97e4-0c5c0478c259 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:21.999447 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e2485-0361-4299-97e4-0c5c0478c259 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:21.999727 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e2485-0361-4299-97e4-0c5c0478c259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:22.032230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:22.078044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-991e2485-0361-4299-97e4-0c5c0478c259 None None] MFA Rules not processed for user `b78747bb9140457896fe62d4f537592c`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:22.079325 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-991e2485-0361-4299-97e4-0c5c0478c259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:22.155756 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 758/1504] 10.222.0.18 () {58 vars in 1061 bytes} [Thu Sep 1 16:30:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:22.165810 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace39f14-9a89-4d25-9274-e2dea5dd7c46 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:22.167428 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ace39f14-9a89-4d25-9274-e2dea5dd7c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:22.219580 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:22.274575 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace39f14-9a89-4d25-9274-e2dea5dd7c46 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:22.349053 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104acf68-9c92-49b9-9616-382bc8945831 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:22.349669 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104acf68-9c92-49b9-9616-382bc8945831 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:22.349948 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104acf68-9c92-49b9-9616-382bc8945831 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:22.350222 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104acf68-9c92-49b9-9616-382bc8945831 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:22.351865 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-104acf68-9c92-49b9-9616-382bc8945831 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:22.456437 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104acf68-9c92-49b9-9616-382bc8945831 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:22.457805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104acf68-9c92-49b9-9616-382bc8945831 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:22.462001 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:22.479024 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 747/1505] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:22 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:22.527992 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4c437-a9cc-4500-a1d6-f3f60bcf73e1 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:22.528445 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4c437-a9cc-4500-a1d6-f3f60bcf73e1 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:22.528564 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4c437-a9cc-4500-a1d6-f3f60bcf73e1 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:22.529836 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 759/1506] 10.222.0.18 () {52 vars in 830 bytes} [Thu Sep 1 16:30:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:30:22.533498 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104acf68-9c92-49b9-9616-382bc8945831 service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:22.534738 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-104acf68-9c92-49b9-9616-382bc8945831 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:22.639028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104acf68-9c92-49b9-9616-382bc8945831 service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:22.732135 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc25e3-b27a-46f9-91c2-be7fa458fb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:22.732706 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc25e3-b27a-46f9-91c2-be7fa458fb1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:22.733239 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc25e3-b27a-46f9-91c2-be7fa458fb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:22.733537 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc25e3-b27a-46f9-91c2-be7fa458fb1a admin admin] PATH_INFO: `/v3/users/e04386f72f1b432aa6f40c0b3d1b9338` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:22.746731 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc25e3-b27a-46f9-91c2-be7fa458fb1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e04386f72f1b432aa6f40c0b3d1b9338)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:22.747752 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: WARNING py.warnings [None req-54bc25e3-b27a-46f9-91c2-be7fa458fb1a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:30:22.747752 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: warnings.warn(msg) Sep 01 16:30:22.747752 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: Sep 01 16:30:22.748968 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc25e3-b27a-46f9-91c2-be7fa458fb1a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:22.758828 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:22.817081 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-54bc25e3-b27a-46f9-91c2-be7fa458fb1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e04386f72f1b432aa6f40c0b3d1b9338'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:22.827714 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-54bc25e3-b27a-46f9-91c2-be7fa458fb1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e04386f72f1b432aa6f40c0b3d1b9338'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:22.828274 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-54bc25e3-b27a-46f9-91c2-be7fa458fb1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e04386f72f1b432aa6f40c0b3d1b9338'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:22.852164 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-54bc25e3-b27a-46f9-91c2-be7fa458fb1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e04386f72f1b432aa6f40c0b3d1b9338'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:22.852652 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-54bc25e3-b27a-46f9-91c2-be7fa458fb1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e04386f72f1b432aa6f40c0b3d1b9338'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:22.872699 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 748/1507] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:30:22 2022] DELETE /identity/v3/users/e04386f72f1b432aa6f40c0b3d1b9338 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:30:22.877307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04137b-f31a-434d-8a40-89b45520ff0d None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:22.877598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04137b-f31a-434d-8a40-89b45520ff0d None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:22.877886 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04137b-f31a-434d-8a40-89b45520ff0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:22.943354 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-5d04137b-f31a-434d-8a40-89b45520ff0d None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:22.944593 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-5d04137b-f31a-434d-8a40-89b45520ff0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:23.021360 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:23.033629 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 760/1508] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:30:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:23.044396 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc25e3-b27a-46f9-91c2-be7fa458fb1a admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:23.045610 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-54bc25e3-b27a-46f9-91c2-be7fa458fb1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:23.144083 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc25e3-b27a-46f9-91c2-be7fa458fb1a admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:23.200865 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:23.220845 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9067ab-284a-40c2-bebb-ee13d43d7c67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:23.221456 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9067ab-284a-40c2-bebb-ee13d43d7c67 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:23.221738 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9067ab-284a-40c2-bebb-ee13d43d7c67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:23.222006 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9067ab-284a-40c2-bebb-ee13d43d7c67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:23.223614 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9a9067ab-284a-40c2-bebb-ee13d43d7c67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:23.331021 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9067ab-284a-40c2-bebb-ee13d43d7c67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:23.332474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9067ab-284a-40c2-bebb-ee13d43d7c67 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:23.350904 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 749/1509] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:23.725488 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04137b-f31a-434d-8a40-89b45520ff0d None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:23.726770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-5d04137b-f31a-434d-8a40-89b45520ff0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:23.738888 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04137b-f31a-434d-8a40-89b45520ff0d None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:23.840212 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecfdf32-d800-4346-a0cc-55285c35afc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:23.840860 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfdf32-d800-4346-a0cc-55285c35afc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:23.841158 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfdf32-d800-4346-a0cc-55285c35afc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:23.841419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfdf32-d800-4346-a0cc-55285c35afc0 admin admin] PATH_INFO: `/v3/projects/a5d6662e6cf24b06a9ec0a5159fbfcd7` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:23.851681 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecfdf32-d800-4346-a0cc-55285c35afc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5d6662e6cf24b06a9ec0a5159fbfcd7)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:23.852651 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: WARNING py.warnings [None req-0ecfdf32-d800-4346-a0cc-55285c35afc0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:30:23.852651 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: warnings.warn(msg) Sep 01 16:30:23.852651 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: Sep 01 16:30:23.853865 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecfdf32-d800-4346-a0cc-55285c35afc0 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:23.911926 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-0ecfdf32-d800-4346-a0cc-55285c35afc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5d6662e6cf24b06a9ec0a5159fbfcd7', 'request_id': 'req-0ecfdf32-d800-4346-a0cc-55285c35afc0'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:23.921905 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-0ecfdf32-d800-4346-a0cc-55285c35afc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5d6662e6cf24b06a9ec0a5159fbfcd7', 'request_id': 'req-0ecfdf32-d800-4346-a0cc-55285c35afc0'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:23.935729 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-0ecfdf32-d800-4346-a0cc-55285c35afc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5d6662e6cf24b06a9ec0a5159fbfcd7', 'request_id': 'req-0ecfdf32-d800-4346-a0cc-55285c35afc0'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:23.958320 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-0ecfdf32-d800-4346-a0cc-55285c35afc0 admin admin] The token cache is being invalidate because project a5d6662e6cf24b06a9ec0a5159fbfcd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:30:23.958618 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-0ecfdf32-d800-4346-a0cc-55285c35afc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:23.959059 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-0ecfdf32-d800-4346-a0cc-55285c35afc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:30:23.960469 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 761/1510] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:30:23 2022] DELETE /identity/v3/projects/a5d6662e6cf24b06a9ec0a5159fbfcd7 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:30:33.348634 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=62 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:33.956170 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:57.290571 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbad3e-5900-4110-b1c9-19e9bd7e568e None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:57.291812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbad3e-5900-4110-b1c9-19e9bd7e568e None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:57.291812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbad3e-5900-4110-b1c9-19e9bd7e568e None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:57.293154 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 750/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:31:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:31:57.297856 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d3601-0f44-4ef9-808a-02efab439053 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:57.298292 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d3601-0f44-4ef9-808a-02efab439053 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:57.298450 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d3601-0f44-4ef9-808a-02efab439053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:57.378006 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-8a1d3601-0f44-4ef9-808a-02efab439053 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:57.379254 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-8a1d3601-0f44-4ef9-808a-02efab439053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:57.485382 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 762/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:31:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:57.639339 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbad3e-5900-4110-b1c9-19e9bd7e568e None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:57.641510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-22dbad3e-5900-4110-b1c9-19e9bd7e568e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:57.758820 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbad3e-5900-4110-b1c9-19e9bd7e568e None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:57.764985 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b25a86c-fa62-4884-8b49-6b50bc3f7f7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:57.765608 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b25a86c-fa62-4884-8b49-6b50bc3f7f7b service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:57.765895 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b25a86c-fa62-4884-8b49-6b50bc3f7f7b service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:57.766166 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b25a86c-fa62-4884-8b49-6b50bc3f7f7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:57.774358 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b25a86c-fa62-4884-8b49-6b50bc3f7f7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:57.776533 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b25a86c-fa62-4884-8b49-6b50bc3f7f7b service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:57.785753 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 751/1513] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:58.606941 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655cd55-9260-4aa6-8d16-7b3427c1f54d None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:58.607766 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655cd55-9260-4aa6-8d16-7b3427c1f54d None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:58.607766 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655cd55-9260-4aa6-8d16-7b3427c1f54d None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:58.609091 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 763/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:31:58.613827 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc559b3-e681-4edc-80a7-5190667358e0 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:58.614223 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc559b3-e681-4edc-80a7-5190667358e0 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:58.614408 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc559b3-e681-4edc-80a7-5190667358e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:58.634286 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-dfc559b3-e681-4edc-80a7-5190667358e0 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:58.635478 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-dfc559b3-e681-4edc-80a7-5190667358e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:58.643153 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 752/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:31:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:58.794467 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655cd55-9260-4aa6-8d16-7b3427c1f54d None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:58.808660 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655cd55-9260-4aa6-8d16-7b3427c1f54d None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:58.814269 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250fc031-1452-4e7d-bd99-186d5494c31c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:58.814873 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fc031-1452-4e7d-bd99-186d5494c31c service nova] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:58.815152 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fc031-1452-4e7d-bd99-186d5494c31c service nova] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:58.815422 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fc031-1452-4e7d-bd99-186d5494c31c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:58.823547 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250fc031-1452-4e7d-bd99-186d5494c31c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:58.825135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250fc031-1452-4e7d-bd99-186d5494c31c service nova] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:58.834286 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 764/1516] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:58.959917 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35779d47-50ae-4b6b-b248-f5b1b113f38a None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:58.960203 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35779d47-50ae-4b6b-b248-f5b1b113f38a None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:58.960483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35779d47-50ae-4b6b-b248-f5b1b113f38a None None] PATH_INFO: `/v3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:58.961770 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 753/1517] 10.222.0.77 () {54 vars in 925 bytes} [Thu Sep 1 16:31:58 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:31:58.968403 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15dd16e-65ba-4554-b176-43ecb1f8b796 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:58.968696 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15dd16e-65ba-4554-b176-43ecb1f8b796 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:58.968979 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15dd16e-65ba-4554-b176-43ecb1f8b796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:59.028209 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-b15dd16e-65ba-4554-b176-43ecb1f8b796 None None] MFA Rules not processed for user `3ddfb151bbf844efae73ec1990e7eaa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:59.029398 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-b15dd16e-65ba-4554-b176-43ecb1f8b796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:59.046320 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:59.099637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 765/1518] 10.222.0.77 () {58 vars in 1015 bytes} [Thu Sep 1 16:31:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:08.280906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa63d8d-3a11-40c8-8e30-67f230c4500a None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:08.281370 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa63d8d-3a11-40c8-8e30-67f230c4500a None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:08.281370 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa63d8d-3a11-40c8-8e30-67f230c4500a None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:08.282750 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 754/1519] 10.222.0.18 () {54 vars in 961 bytes} [Thu Sep 1 16:32:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:32:08.288885 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d63aa-3003-4751-a0c4-0883a0983fb2 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:08.289307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d63aa-3003-4751-a0c4-0883a0983fb2 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:08.289445 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d63aa-3003-4751-a0c4-0883a0983fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:08.309993 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-a39d63aa-3003-4751-a0c4-0883a0983fb2 None None] MFA Rules not processed for user `3ddfb151bbf844efae73ec1990e7eaa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:08.311220 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-a39d63aa-3003-4751-a0c4-0883a0983fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:08.319556 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 766/1520] 10.222.0.18 () {58 vars in 1058 bytes} [Thu Sep 1 16:32:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:32:08.326810 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa63d8d-3a11-40c8-8e30-67f230c4500a None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:08.339804 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa63d8d-3a11-40c8-8e30-67f230c4500a None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:08.344690 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a25e5-1326-40c1-884f-4b05eacd28ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ddfb151bbf844efae73ec1990e7eaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:08.345274 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a25e5-1326-40c1-884f-4b05eacd28ce service placement] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:08.345557 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a25e5-1326-40c1-884f-4b05eacd28ce service placement] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:08.345829 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a25e5-1326-40c1-884f-4b05eacd28ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:08.354973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75a25e5-1326-40c1-884f-4b05eacd28ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:08.356554 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75a25e5-1326-40c1-884f-4b05eacd28ce service placement] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:08.366658 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 755/1521] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:32:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:32:10.056378 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af53527-2635-4dbc-99a6-3353eb0dacc5 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:10.056956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af53527-2635-4dbc-99a6-3353eb0dacc5 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:10.056956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af53527-2635-4dbc-99a6-3353eb0dacc5 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:10.058858 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 767/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:32:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:32:10.063626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da53f6-191f-491b-9734-854510332ea7 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:10.064029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da53f6-191f-491b-9734-854510332ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:10.064182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da53f6-191f-491b-9734-854510332ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:10.083526 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-e7da53f6-191f-491b-9734-854510332ea7 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:10.084708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-e7da53f6-191f-491b-9734-854510332ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:10.092443 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 756/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:10.245598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af53527-2635-4dbc-99a6-3353eb0dacc5 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:10.260113 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af53527-2635-4dbc-99a6-3353eb0dacc5 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:10.265444 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fbacf4-432a-4027-9827-a84532fc49e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:10.266037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbacf4-432a-4027-9827-a84532fc49e9 service nova] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:10.266364 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbacf4-432a-4027-9827-a84532fc49e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:10.266643 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbacf4-432a-4027-9827-a84532fc49e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:10.274770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fbacf4-432a-4027-9827-a84532fc49e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:10.276928 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fbacf4-432a-4027-9827-a84532fc49e9 service nova] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:10.286211 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 768/1524] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:32:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:32:11.112792 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8e080-91d5-4d49-b496-3b2619ec400c None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:11.113340 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8e080-91d5-4d49-b496-3b2619ec400c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:11.113340 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8e080-91d5-4d49-b496-3b2619ec400c None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:11.114750 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 757/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:32:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:32:11.119652 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c796136c-5496-46f5-bd0a-90129a63e5c7 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:11.120703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c796136c-5496-46f5-bd0a-90129a63e5c7 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:11.120851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c796136c-5496-46f5-bd0a-90129a63e5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:11.139939 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-c796136c-5496-46f5-bd0a-90129a63e5c7 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:11.141142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-c796136c-5496-46f5-bd0a-90129a63e5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:11.148924 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 769/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:11.300469 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb8e080-91d5-4d49-b496-3b2619ec400c None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:11.315328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb8e080-91d5-4d49-b496-3b2619ec400c None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:11.320666 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eae3f7-38c3-4951-9ada-f3b41dd47824 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:11.321261 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eae3f7-38c3-4951-9ada-f3b41dd47824 service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:11.321544 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eae3f7-38c3-4951-9ada-f3b41dd47824 service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:11.321818 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eae3f7-38c3-4951-9ada-f3b41dd47824 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:11.329653 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eae3f7-38c3-4951-9ada-f3b41dd47824 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:11.331268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eae3f7-38c3-4951-9ada-f3b41dd47824 service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:11.340454 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 758/1527] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:32:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:32:15.407056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187fd41-4786-476c-9bb0-212da40e1010 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:15.407553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187fd41-4786-476c-9bb0-212da40e1010 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:15.407666 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187fd41-4786-476c-9bb0-212da40e1010 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:15.408956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 770/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:32:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:32:15.413982 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897d01b-88a2-4a10-93d9-e48e4171d64d None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:15.414399 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897d01b-88a2-4a10-93d9-e48e4171d64d None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:15.414571 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897d01b-88a2-4a10-93d9-e48e4171d64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:15.433998 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-f897d01b-88a2-4a10-93d9-e48e4171d64d None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:15.435206 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f897d01b-88a2-4a10-93d9-e48e4171d64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:15.442850 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 759/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:15.727865 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f187fd41-4786-476c-9bb0-212da40e1010 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:15.730147 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-f187fd41-4786-476c-9bb0-212da40e1010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:15.808155 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f187fd41-4786-476c-9bb0-212da40e1010 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:15.811737 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c453e-1676-4458-890b-0398e7483759 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:15.812342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c453e-1676-4458-890b-0398e7483759 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:15.812627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c453e-1676-4458-890b-0398e7483759 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:15.812896 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c453e-1676-4458-890b-0398e7483759 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:15.821725 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40c453e-1676-4458-890b-0398e7483759 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:15.823270 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40c453e-1676-4458-890b-0398e7483759 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:15.832512 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 771/1530] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:32:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:32:17.700653 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae33a8-88b3-4d4d-99d8-136a51bed900 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:17.701176 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae33a8-88b3-4d4d-99d8-136a51bed900 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:17.701176 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae33a8-88b3-4d4d-99d8-136a51bed900 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:17.702492 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 760/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:32:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:32:17.707613 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e6151-5dde-42a2-a81f-2a026fcd99d9 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:17.708053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e6151-5dde-42a2-a81f-2a026fcd99d9 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:17.708235 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e6151-5dde-42a2-a81f-2a026fcd99d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:17.727959 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-5c2e6151-5dde-42a2-a81f-2a026fcd99d9 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:17.729155 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-5c2e6151-5dde-42a2-a81f-2a026fcd99d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:17.737764 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 772/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:32:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:18.024779 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae33a8-88b3-4d4d-99d8-136a51bed900 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:18.036033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae33a8-88b3-4d4d-99d8-136a51bed900 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:18.039514 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f92a1a9-ea7e-498e-bc5a-2aee140f4ce0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:18.040108 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92a1a9-ea7e-498e-bc5a-2aee140f4ce0 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:18.040391 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92a1a9-ea7e-498e-bc5a-2aee140f4ce0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:18.040664 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92a1a9-ea7e-498e-bc5a-2aee140f4ce0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:18.049252 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f92a1a9-ea7e-498e-bc5a-2aee140f4ce0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:18.050721 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f92a1a9-ea7e-498e-bc5a-2aee140f4ce0 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:18.059792 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 761/1533] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:32:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:32:21.989491 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ed1b5-1234-4073-8981-e30a02d09d6e None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:21.990060 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ed1b5-1234-4073-8981-e30a02d09d6e None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:21.990060 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ed1b5-1234-4073-8981-e30a02d09d6e None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:21.991431 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 773/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:32:21.997282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd7752-5857-4f93-bb90-396cd6474c13 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:21.997720 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd7752-5857-4f93-bb90-396cd6474c13 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:21.997837 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd7752-5857-4f93-bb90-396cd6474c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:22.017786 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-f6bd7752-5857-4f93-bb90-396cd6474c13 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:22.019027 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f6bd7752-5857-4f93-bb90-396cd6474c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:22.026787 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 762/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:22.573449 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ed1b5-1234-4073-8981-e30a02d09d6e None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:22.575446 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-9f2ed1b5-1234-4073-8981-e30a02d09d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:22.640685 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:22.654867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ed1b5-1234-4073-8981-e30a02d09d6e None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:22.658427 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844a320-c72b-4e12-a248-754238ef515a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:22.659028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844a320-c72b-4e12-a248-754238ef515a service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:22.659309 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844a320-c72b-4e12-a248-754238ef515a service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:22.659583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844a320-c72b-4e12-a248-754238ef515a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:22.668340 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844a320-c72b-4e12-a248-754238ef515a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:22.669796 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844a320-c72b-4e12-a248-754238ef515a service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:22.679033 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 774/1536] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:32:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:32:24.578082 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11767d8-2051-4e03-8ec0-6604dc705a2d None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:24.578739 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11767d8-2051-4e03-8ec0-6604dc705a2d None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:24.578739 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11767d8-2051-4e03-8ec0-6604dc705a2d None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:24.579988 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 763/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:32:24.585085 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580dfbe-3e61-49c6-a3f7-e84ca4834646 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:24.585525 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580dfbe-3e61-49c6-a3f7-e84ca4834646 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:24.585629 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580dfbe-3e61-49c6-a3f7-e84ca4834646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:24.606704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-f580dfbe-3e61-49c6-a3f7-e84ca4834646 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:24.608147 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-f580dfbe-3e61-49c6-a3f7-e84ca4834646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:24.615933 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 775/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:25.165902 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11767d8-2051-4e03-8ec0-6604dc705a2d None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:25.168405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b11767d8-2051-4e03-8ec0-6604dc705a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:25.175830 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=84 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:25.182495 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11767d8-2051-4e03-8ec0-6604dc705a2d None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:25.186052 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54280c61-b47a-4625-88c2-1820b2dd2db0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:25.186698 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54280c61-b47a-4625-88c2-1820b2dd2db0 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:25.186979 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54280c61-b47a-4625-88c2-1820b2dd2db0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:25.187252 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54280c61-b47a-4625-88c2-1820b2dd2db0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:25.202758 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54280c61-b47a-4625-88c2-1820b2dd2db0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:25.205107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54280c61-b47a-4625-88c2-1820b2dd2db0 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:25.215363 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 764/1539] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:32:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:32:34.600619 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=11 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:35.210645 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=5 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:50.004352 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fe30a-764c-40c8-aa90-9081ec8d6835 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:50.005487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fe30a-764c-40c8-aa90-9081ec8d6835 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:50.005991 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fe30a-764c-40c8-aa90-9081ec8d6835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:50.039418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-6e2fe30a-764c-40c8-aa90-9081ec8d6835 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:50.040673 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-6e2fe30a-764c-40c8-aa90-9081ec8d6835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:50.057332 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 776/1540] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:50.067021 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c28a8-0684-41e1-862a-4498e475219d None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:50.067627 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c28a8-0684-41e1-862a-4498e475219d None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:50.067627 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c28a8-0684-41e1-862a-4498e475219d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:50.098706 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-558c28a8-0684-41e1-862a-4498e475219d None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:50.101487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-558c28a8-0684-41e1-862a-4498e475219d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:50.119601 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 765/1541] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:50.124807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fe30a-764c-40c8-aa90-9081ec8d6835 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:50.141774 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fe30a-764c-40c8-aa90-9081ec8d6835 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:50.153960 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ac9ca-2a18-4bc6-9687-340179f542d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:50.154650 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ac9ca-2a18-4bc6-9687-340179f542d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:50.154929 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ac9ca-2a18-4bc6-9687-340179f542d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:50.155200 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ac9ca-2a18-4bc6-9687-340179f542d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:50.155888 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ac9ca-2a18-4bc6-9687-340179f542d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:33:50.156192 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ac9ca-2a18-4bc6-9687-340179f542d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:50.157927 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ac9ca-2a18-4bc6-9687-340179f542d7 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:50.178927 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 777/1542] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:33:50.187765 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c28a8-0684-41e1-862a-4498e475219d None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:50.197937 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c28a8-0684-41e1-862a-4498e475219d None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:50.209341 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a304e0-b577-4679-bb74-ef4c9ed207e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:50.209985 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a304e0-b577-4679-bb74-ef4c9ed207e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:50.210284 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a304e0-b577-4679-bb74-ef4c9ed207e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:50.210566 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a304e0-b577-4679-bb74-ef4c9ed207e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:50.211223 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a304e0-b577-4679-bb74-ef4c9ed207e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:50.213147 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a304e0-b577-4679-bb74-ef4c9ed207e9 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:50.264547 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 766/1543] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:33:50 2022] POST /identity/v3/projects => generated 367 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:50.268893 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ac9ca-2a18-4bc6-9687-340179f542d7 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:50.290769 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ac9ca-2a18-4bc6-9687-340179f542d7 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:50.350226 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9653f169-c3bf-46c1-964c-d35c6ab543f7 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:50.350226 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9653f169-c3bf-46c1-964c-d35c6ab543f7 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:50.350226 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9653f169-c3bf-46c1-964c-d35c6ab543f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:50.380122 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd374ae4-1c63-41a8-876f-b7d43e9f81ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:50.380747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd374ae4-1c63-41a8-876f-b7d43e9f81ec admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:50.381025 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd374ae4-1c63-41a8-876f-b7d43e9f81ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:50.381365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd374ae4-1c63-41a8-876f-b7d43e9f81ec admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:50.382007 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd374ae4-1c63-41a8-876f-b7d43e9f81ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:50.383990 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd374ae4-1c63-41a8-876f-b7d43e9f81ec admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:50.385648 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-9653f169-c3bf-46c1-964c-d35c6ab543f7 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:50.386920 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9653f169-c3bf-46c1-964c-d35c6ab543f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:50.406759 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 767/1544] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 01 16:33:50.408131 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5ae26-2891-450a-af3a-687756d27747 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:50.409591 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5ae26-2891-450a-af3a-687756d27747 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:50.409882 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5ae26-2891-450a-af3a-687756d27747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:50.434096 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 778/1545] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:33:50 2022] POST /identity/v3/users => generated 284 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:50.438628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd374ae4-1c63-41a8-876f-b7d43e9f81ec admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:50.459065 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-84d5ae26-2891-450a-af3a-687756d27747 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:50.460690 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-84d5ae26-2891-450a-af3a-687756d27747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:50.464269 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd374ae4-1c63-41a8-876f-b7d43e9f81ec admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:50.475749 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122db84d-a61d-46bf-af12-db524303d7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:50.476441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122db84d-a61d-46bf-af12-db524303d7f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:50.476777 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122db84d-a61d-46bf-af12-db524303d7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:50.477056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122db84d-a61d-46bf-af12-db524303d7f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:50.477714 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122db84d-a61d-46bf-af12-db524303d7f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:50.478668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 768/1546] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:50.480378 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122db84d-a61d-46bf-af12-db524303d7f9 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:50.480990 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab260a61-b790-4f56-b4da-e21d605fa0df None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:50.481998 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab260a61-b790-4f56-b4da-e21d605fa0df None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:50.482518 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab260a61-b790-4f56-b4da-e21d605fa0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:50.504923 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 779/1547] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:33:50.506313 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0dc20-1744-44ab-a7ee-1b6cb7bf771e None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:50.506604 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0dc20-1744-44ab-a7ee-1b6cb7bf771e None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:50.506880 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0dc20-1744-44ab-a7ee-1b6cb7bf771e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:50.527364 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-ab260a61-b790-4f56-b4da-e21d605fa0df None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:50.529886 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ab260a61-b790-4f56-b4da-e21d605fa0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:50.541914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 769/1548] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:50.542646 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab260a61-b790-4f56-b4da-e21d605fa0df None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:50.544416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-88e0dc20-1744-44ab-a7ee-1b6cb7bf771e None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:50.546075 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-88e0dc20-1744-44ab-a7ee-1b6cb7bf771e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:50.555814 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 780/1549] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:50.556424 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab260a61-b790-4f56-b4da-e21d605fa0df None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:50.557498 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec286d-94fc-4738-9de3-225dfb9d3c79 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:50.557821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec286d-94fc-4738-9de3-225dfb9d3c79 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:50.558125 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec286d-94fc-4738-9de3-225dfb9d3c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:50.563681 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995cde8d-8e11-41e2-b40c-8b4cff16ffdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:50.564294 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cde8d-8e11-41e2-b40c-8b4cff16ffdc admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:50.564639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cde8d-8e11-41e2-b40c-8b4cff16ffdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:50.564833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cde8d-8e11-41e2-b40c-8b4cff16ffdc admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:50.565471 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995cde8d-8e11-41e2-b40c-8b4cff16ffdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:50.568275 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995cde8d-8e11-41e2-b40c-8b4cff16ffdc admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:50.578764 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-b9ec286d-94fc-4738-9de3-225dfb9d3c79 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:50.579594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 770/1550] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:50.580063 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-b9ec286d-94fc-4738-9de3-225dfb9d3c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:50.580647 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19166b7-ffa7-466d-9e4e-59b1152b5e50 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:50.580935 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19166b7-ffa7-466d-9e4e-59b1152b5e50 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:50.581211 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19166b7-ffa7-466d-9e4e-59b1152b5e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:50.588892 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 781/1551] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:50.590106 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16efb7-6f87-4301-bfe1-965290f7843c None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:50.590922 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16efb7-6f87-4301-bfe1-965290f7843c None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:50.591249 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16efb7-6f87-4301-bfe1-965290f7843c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:50.612212 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-f19166b7-ffa7-466d-9e4e-59b1152b5e50 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:50.612212 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f19166b7-ffa7-466d-9e4e-59b1152b5e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:50.613898 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-ba16efb7-6f87-4301-bfe1-965290f7843c None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:50.615334 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-ba16efb7-6f87-4301-bfe1-965290f7843c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:50.618747 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 771/1552] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:50.620038 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f55cba9-8900-4395-8ee4-5a53b76927bc None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:50.620324 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f55cba9-8900-4395-8ee4-5a53b76927bc None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:50.620598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f55cba9-8900-4395-8ee4-5a53b76927bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:50.626497 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 782/1553] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:50.627464 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16efb7-6f87-4301-bfe1-965290f7843c None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:50.638615 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16efb7-6f87-4301-bfe1-965290f7843c None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:50.640226 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-8f55cba9-8900-4395-8ee4-5a53b76927bc None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:50.641459 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8f55cba9-8900-4395-8ee4-5a53b76927bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:50.643515 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd85db9-61ac-4183-b139-e74ad62541d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:50.644114 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd85db9-61ac-4183-b139-e74ad62541d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:50.644395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd85db9-61ac-4183-b139-e74ad62541d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:50.644669 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd85db9-61ac-4183-b139-e74ad62541d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:50.645310 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd85db9-61ac-4183-b139-e74ad62541d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:33:50.645621 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd85db9-61ac-4183-b139-e74ad62541d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:50.647240 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd85db9-61ac-4183-b139-e74ad62541d9 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:50.649446 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 772/1554] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:50.650219 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f55cba9-8900-4395-8ee4-5a53b76927bc None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:50.659967 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f55cba9-8900-4395-8ee4-5a53b76927bc None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:50.664531 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf8bf1-8825-4dcb-a601-235ce2e20c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:50.672460 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf8bf1-8825-4dcb-a601-235ce2e20c09 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:50.672665 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 783/1555] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:33:50.672771 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf8bf1-8825-4dcb-a601-235ce2e20c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:50.672771 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf8bf1-8825-4dcb-a601-235ce2e20c09 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:50.672969 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd85db9-61ac-4183-b139-e74ad62541d9 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:50.673060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccf8bf1-8825-4dcb-a601-235ce2e20c09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:33:50.673060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccf8bf1-8825-4dcb-a601-235ce2e20c09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:50.673060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: WARNING py.warnings [None req-dccf8bf1-8825-4dcb-a601-235ce2e20c09 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:33:50.673060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: warnings.warn(msg) Sep 01 16:33:50.673060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: Sep 01 16:33:50.673060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccf8bf1-8825-4dcb-a601-235ce2e20c09 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:50.676112 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd85db9-61ac-4183-b139-e74ad62541d9 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:50.680724 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780cc26-ad02-4ea9-a431-0d8512c04fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:50.681363 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780cc26-ad02-4ea9-a431-0d8512c04fe9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:50.681642 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780cc26-ad02-4ea9-a431-0d8512c04fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:50.682101 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780cc26-ad02-4ea9-a431-0d8512c04fe9 admin admin] PATH_INFO: `/v3/projects/07369ffa37c54449978dfb5ef24150ed/users/546082aeb45e4ebb9b4619feb41b50fd/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:50.682229 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 773/1556] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:33:50.683206 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c59616-b81d-4a7b-b679-f14ae4a8bb4e None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:50.683497 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c59616-b81d-4a7b-b679-f14ae4a8bb4e None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:50.683780 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c59616-b81d-4a7b-b679-f14ae4a8bb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:50.698408 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780cc26-ad02-4ea9-a431-0d8512c04fe9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07369ffa37c54449978dfb5ef24150ed, user_id=546082aeb45e4ebb9b4619feb41b50fd, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:50.699448 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: WARNING py.warnings [None req-a780cc26-ad02-4ea9-a431-0d8512c04fe9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:33:50.699448 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: warnings.warn(msg) Sep 01 16:33:50.699448 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: Sep 01 16:33:50.701930 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780cc26-ad02-4ea9-a431-0d8512c04fe9 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:50.703596 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-a3c59616-b81d-4a7b-b679-f14ae4a8bb4e None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:50.704808 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a3c59616-b81d-4a7b-b679-f14ae4a8bb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:50.715058 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 774/1557] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:50.717456 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14c4d7-5e3d-4f69-952b-9e9ca61daa2e None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:50.717456 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14c4d7-5e3d-4f69-952b-9e9ca61daa2e None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:50.717456 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14c4d7-5e3d-4f69-952b-9e9ca61daa2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:50.718802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 784/1558] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:50 2022] PUT /identity/v3/projects/07369ffa37c54449978dfb5ef24150ed/users/546082aeb45e4ebb9b4619feb41b50fd/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:50.720976 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f5940-57fc-4161-ad74-05745ec2fb20 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:50.721314 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f5940-57fc-4161-ad74-05745ec2fb20 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:50.721543 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f5940-57fc-4161-ad74-05745ec2fb20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:50.737475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-7d14c4d7-5e3d-4f69-952b-9e9ca61daa2e None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:50.738710 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7d14c4d7-5e3d-4f69-952b-9e9ca61daa2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:50.741718 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-ce0f5940-57fc-4161-ad74-05745ec2fb20 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:50.742919 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-ce0f5940-57fc-4161-ad74-05745ec2fb20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:50.808431 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 775/1559] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:50.810101 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391e9ff-7950-432a-b1e1-d89e590de0e7 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:50.810575 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391e9ff-7950-432a-b1e1-d89e590de0e7 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:50.811030 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391e9ff-7950-432a-b1e1-d89e590de0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:50.819692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 785/1560] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:50.821510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38aaa51-fb90-4a46-a9cd-1b988406826e None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:50.821510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38aaa51-fb90-4a46-a9cd-1b988406826e None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:50.821704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38aaa51-fb90-4a46-a9cd-1b988406826e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:50.830214 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:50.831331 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-4391e9ff-7950-432a-b1e1-d89e590de0e7 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:50.832547 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-4391e9ff-7950-432a-b1e1-d89e590de0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:50.841042 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 776/1561] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:50.841811 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391e9ff-7950-432a-b1e1-d89e590de0e7 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:50.844504 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-a38aaa51-fb90-4a46-a9cd-1b988406826e None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:50.845984 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-a38aaa51-fb90-4a46-a9cd-1b988406826e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:50.853935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 786/1562] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:50.854552 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391e9ff-7950-432a-b1e1-d89e590de0e7 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:50.854736 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38aaa51-fb90-4a46-a9cd-1b988406826e None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:50.859642 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b9fb7-cb62-4d13-9333-df6de1bdf40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:50.860245 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b9fb7-cb62-4d13-9333-df6de1bdf40a admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:50.860535 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b9fb7-cb62-4d13-9333-df6de1bdf40a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:50.860810 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b9fb7-cb62-4d13-9333-df6de1bdf40a admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:50.861438 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41b9fb7-cb62-4d13-9333-df6de1bdf40a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:50.863411 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41b9fb7-cb62-4d13-9333-df6de1bdf40a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:50.865241 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38aaa51-fb90-4a46-a9cd-1b988406826e None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:50.871476 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284dbec9-289a-4d5d-b5a4-9440df0546aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:50.872079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284dbec9-289a-4d5d-b5a4-9440df0546aa admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:50.872365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284dbec9-289a-4d5d-b5a4-9440df0546aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:50.872649 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284dbec9-289a-4d5d-b5a4-9440df0546aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:50.873284 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284dbec9-289a-4d5d-b5a4-9440df0546aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:50.874349 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: WARNING py.warnings [None req-284dbec9-289a-4d5d-b5a4-9440df0546aa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:33:50.874349 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: warnings.warn(msg) Sep 01 16:33:50.874349 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: Sep 01 16:33:50.875558 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284dbec9-289a-4d5d-b5a4-9440df0546aa admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:50.891974 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 777/1563] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:33:50 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:50.892882 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b9fb7-cb62-4d13-9333-df6de1bdf40a admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:50.911590 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 787/1564] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:33:50 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:50.911590 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddb720-307e-4a34-8542-4bd0767bd472 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:50.911590 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddb720-307e-4a34-8542-4bd0767bd472 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:50.911590 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddb720-307e-4a34-8542-4bd0767bd472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:50.931327 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:50.936668 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-0dddb720-307e-4a34-8542-4bd0767bd472 None None] MFA Rules not processed for user `546082aeb45e4ebb9b4619feb41b50fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:50.937864 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-0dddb720-307e-4a34-8542-4bd0767bd472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:50.960836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b9fb7-cb62-4d13-9333-df6de1bdf40a admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:51.007054 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 788/1565] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:33:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:51.008307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881baaba-34c8-47f3-9e71-28167b207cdc None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:51.008595 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881baaba-34c8-47f3-9e71-28167b207cdc None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:51.008878 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881baaba-34c8-47f3-9e71-28167b207cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:51.024178 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21caa386-6ad7-48ed-81e5-8c3afbe5e991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:51.024796 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21caa386-6ad7-48ed-81e5-8c3afbe5e991 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:51.025076 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21caa386-6ad7-48ed-81e5-8c3afbe5e991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:51.025347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21caa386-6ad7-48ed-81e5-8c3afbe5e991 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:51.025987 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21caa386-6ad7-48ed-81e5-8c3afbe5e991 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:33:51.026325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21caa386-6ad7-48ed-81e5-8c3afbe5e991 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:51.028604 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21caa386-6ad7-48ed-81e5-8c3afbe5e991 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:51.029132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-881baaba-34c8-47f3-9e71-28167b207cdc None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:51.030352 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-881baaba-34c8-47f3-9e71-28167b207cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:51.038081 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 789/1566] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:51.040440 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 778/1567] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:33:51.041163 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21caa386-6ad7-48ed-81e5-8c3afbe5e991 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:51.042448 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881baaba-34c8-47f3-9e71-28167b207cdc None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:51.056344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21caa386-6ad7-48ed-81e5-8c3afbe5e991 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:51.056956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881baaba-34c8-47f3-9e71-28167b207cdc None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:51.061437 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffeaa63-3a72-45f2-9f11-31fd88170f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:51.061973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffeaa63-3a72-45f2-9f11-31fd88170f7b admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:51.062194 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d405d35-ad17-49b5-82e8-83b45d30d9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:51.062343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffeaa63-3a72-45f2-9f11-31fd88170f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:51.062545 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffeaa63-3a72-45f2-9f11-31fd88170f7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:51.062823 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d405d35-ad17-49b5-82e8-83b45d30d9ea admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:51.063124 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d405d35-ad17-49b5-82e8-83b45d30d9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:51.063284 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffeaa63-3a72-45f2-9f11-31fd88170f7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:33:51.063385 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d405d35-ad17-49b5-82e8-83b45d30d9ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:51.063589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffeaa63-3a72-45f2-9f11-31fd88170f7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:51.064017 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d405d35-ad17-49b5-82e8-83b45d30d9ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:33:51.064319 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d405d35-ad17-49b5-82e8-83b45d30d9ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:51.065155 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffeaa63-3a72-45f2-9f11-31fd88170f7b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:51.065904 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d405d35-ad17-49b5-82e8-83b45d30d9ea admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:51.078581 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 779/1568] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:33:51.079132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 790/1569] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:33:51.079395 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffeaa63-3a72-45f2-9f11-31fd88170f7b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:51.079794 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d405d35-ad17-49b5-82e8-83b45d30d9ea admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:51.093258 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d405d35-ad17-49b5-82e8-83b45d30d9ea admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:51.093481 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffeaa63-3a72-45f2-9f11-31fd88170f7b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:51.097514 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4517398-00da-409f-aa9a-a85edc0f7ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:51.097725 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a46b0-5691-496d-a6aa-e1cdc10683d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:51.098112 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4517398-00da-409f-aa9a-a85edc0f7ca1 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:51.098348 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a46b0-5691-496d-a6aa-e1cdc10683d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:51.098461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4517398-00da-409f-aa9a-a85edc0f7ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:51.098631 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a46b0-5691-496d-a6aa-e1cdc10683d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:51.098747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4517398-00da-409f-aa9a-a85edc0f7ca1 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:51.098901 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a46b0-5691-496d-a6aa-e1cdc10683d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:51.099293 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4517398-00da-409f-aa9a-a85edc0f7ca1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:51.099541 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43a46b0-5691-496d-a6aa-e1cdc10683d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:33:51.099842 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43a46b0-5691-496d-a6aa-e1cdc10683d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:51.101220 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4517398-00da-409f-aa9a-a85edc0f7ca1 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:51.101389 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43a46b0-5691-496d-a6aa-e1cdc10683d7 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:51.118407 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 780/1570] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:33:51.120009 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a46b0-5691-496d-a6aa-e1cdc10683d7 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:51.132359 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 791/1571] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:33:51 2022] POST /identity/v3/users => generated 290 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:51.132359 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4517398-00da-409f-aa9a-a85edc0f7ca1 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:51.137961 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a46b0-5691-496d-a6aa-e1cdc10683d7 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:51.143218 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ff3a9-637a-4463-9db5-2a93c6fe87fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:51.143823 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ff3a9-637a-4463-9db5-2a93c6fe87fc admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:51.144105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ff3a9-637a-4463-9db5-2a93c6fe87fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:51.144382 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ff3a9-637a-4463-9db5-2a93c6fe87fc admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:51.145008 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ff3a9-637a-4463-9db5-2a93c6fe87fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:51.146019 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: WARNING py.warnings [None req-a50ff3a9-637a-4463-9db5-2a93c6fe87fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:33:51.146019 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: warnings.warn(msg) Sep 01 16:33:51.146019 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: Sep 01 16:33:51.147275 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ff3a9-637a-4463-9db5-2a93c6fe87fc admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:51.168661 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 781/1572] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:33:51 2022] POST /identity/v3/users => generated 290 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:51.169608 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ff3a9-637a-4463-9db5-2a93c6fe87fc admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:51.181354 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ff3a9-637a-4463-9db5-2a93c6fe87fc admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:51.187274 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b10900-394c-451a-8669-b7b6a5ec68e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:51.187870 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b10900-394c-451a-8669-b7b6a5ec68e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:51.188153 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b10900-394c-451a-8669-b7b6a5ec68e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:51.188426 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b10900-394c-451a-8669-b7b6a5ec68e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:51.188596 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4517398-00da-409f-aa9a-a85edc0f7ca1 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:51.189068 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b10900-394c-451a-8669-b7b6a5ec68e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:33:51.189371 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b10900-394c-451a-8669-b7b6a5ec68e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:51.190994 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b10900-394c-451a-8669-b7b6a5ec68e5 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:51.192332 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2261ca-2801-4b28-85c7-d46606c1e8d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:51.192937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2261ca-2801-4b28-85c7-d46606c1e8d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:51.193241 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2261ca-2801-4b28-85c7-d46606c1e8d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:51.193511 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2261ca-2801-4b28-85c7-d46606c1e8d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:51.202414 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2261ca-2801-4b28-85c7-d46606c1e8d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:51.203846 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2261ca-2801-4b28-85c7-d46606c1e8d1 service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:51.204167 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 782/1573] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:33:51.205011 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b10900-394c-451a-8669-b7b6a5ec68e5 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:51.216368 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b10900-394c-451a-8669-b7b6a5ec68e5 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:51.218476 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 792/1574] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:51.219341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2261ca-2801-4b28-85c7-d46606c1e8d1 service cinder] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:51.221320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce59e7f-5848-4005-b1f5-c11019c54768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:51.222037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce59e7f-5848-4005-b1f5-c11019c54768 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:51.222414 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce59e7f-5848-4005-b1f5-c11019c54768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:51.222755 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce59e7f-5848-4005-b1f5-c11019c54768 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:51.223450 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce59e7f-5848-4005-b1f5-c11019c54768 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:51.226418 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce59e7f-5848-4005-b1f5-c11019c54768 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:51.229592 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2261ca-2801-4b28-85c7-d46606c1e8d1 service cinder] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:51.236088 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205dac9-52af-412b-8722-344ddebad407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:51.236692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205dac9-52af-412b-8722-344ddebad407 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:51.237093 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205dac9-52af-412b-8722-344ddebad407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:51.237376 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205dac9-52af-412b-8722-344ddebad407 admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:51.238108 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205dac9-52af-412b-8722-344ddebad407 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:51.240343 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205dac9-52af-412b-8722-344ddebad407 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:51.260570 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 783/1575] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:33:51 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:51.261401 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce59e7f-5848-4005-b1f5-c11019c54768 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:51.279526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 793/1576] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:33:51 2022] POST /identity/v3/projects => generated 393 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:51.283881 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205dac9-52af-412b-8722-344ddebad407 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:51.295380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:51.322448 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:51.336760 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce59e7f-5848-4005-b1f5-c11019c54768 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:51.361687 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205dac9-52af-412b-8722-344ddebad407 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:51.370595 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe4693-42b4-4c5a-a058-f95a79571e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:51.371312 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe4693-42b4-4c5a-a058-f95a79571e69 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:51.372532 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe4693-42b4-4c5a-a058-f95a79571e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:51.372532 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe4693-42b4-4c5a-a058-f95a79571e69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:51.373242 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fe4693-42b4-4c5a-a058-f95a79571e69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:51.375401 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fe4693-42b4-4c5a-a058-f95a79571e69 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:51.406090 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 794/1577] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:33:51 2022] POST /identity/v3/projects => generated 377 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:51.407606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe4693-42b4-4c5a-a058-f95a79571e69 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:51.409594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafad52c-215f-40dc-a31d-7d63a11c0036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:51.410219 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafad52c-215f-40dc-a31d-7d63a11c0036 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:51.410531 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafad52c-215f-40dc-a31d-7d63a11c0036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:51.410804 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafad52c-215f-40dc-a31d-7d63a11c0036 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:51.411433 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafad52c-215f-40dc-a31d-7d63a11c0036 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:51.413469 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafad52c-215f-40dc-a31d-7d63a11c0036 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:51.445933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 784/1578] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:33:51 2022] POST /identity/v3/projects => generated 389 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:51.446803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafad52c-215f-40dc-a31d-7d63a11c0036 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:51.479200 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe4693-42b4-4c5a-a058-f95a79571e69 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:51.530720 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafad52c-215f-40dc-a31d-7d63a11c0036 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:51.537011 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe7195-a54e-4ac7-9156-751d85b15d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:51.537974 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe7195-a54e-4ac7-9156-751d85b15d8b admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:51.538112 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe7195-a54e-4ac7-9156-751d85b15d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:51.538314 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe7195-a54e-4ac7-9156-751d85b15d8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:51.538947 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe7195-a54e-4ac7-9156-751d85b15d8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:51.540822 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe7195-a54e-4ac7-9156-751d85b15d8b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:51.553599 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 785/1579] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:51.554535 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe7195-a54e-4ac7-9156-751d85b15d8b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:51.557483 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c937e8-1958-423c-b041-2f3712107fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:51.558387 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c937e8-1958-423c-b041-2f3712107fca admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:51.560083 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c937e8-1958-423c-b041-2f3712107fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:51.560666 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c937e8-1958-423c-b041-2f3712107fca admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:51.562344 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c937e8-1958-423c-b041-2f3712107fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:51.565384 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe7195-a54e-4ac7-9156-751d85b15d8b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:51.565681 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c937e8-1958-423c-b041-2f3712107fca admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:51.570935 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db0fbb9-3b02-4043-97eb-9a5f9735dd1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:51.571544 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0fbb9-3b02-4043-97eb-9a5f9735dd1a admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:51.572638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0fbb9-3b02-4043-97eb-9a5f9735dd1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:51.572877 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0fbb9-3b02-4043-97eb-9a5f9735dd1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:51.573588 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db0fbb9-3b02-4043-97eb-9a5f9735dd1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:51.575735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db0fbb9-3b02-4043-97eb-9a5f9735dd1a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:51.577184 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 795/1580] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:51.577962 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c937e8-1958-423c-b041-2f3712107fca admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:51.590011 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c937e8-1958-423c-b041-2f3712107fca admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:51.595496 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb613685-0320-43f9-9a33-0cc9bf174961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:51.596550 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb613685-0320-43f9-9a33-0cc9bf174961 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:51.596867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb613685-0320-43f9-9a33-0cc9bf174961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:51.597361 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb613685-0320-43f9-9a33-0cc9bf174961 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:51.598231 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb613685-0320-43f9-9a33-0cc9bf174961 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:51.600212 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb613685-0320-43f9-9a33-0cc9bf174961 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:51.605598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 786/1581] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:33:51 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:51.607135 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db0fbb9-3b02-4043-97eb-9a5f9735dd1a admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:51.629020 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 796/1582] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:33:51 2022] POST /identity/v3/users => generated 293 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:51.629912 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb613685-0320-43f9-9a33-0cc9bf174961 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:51.639997 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb613685-0320-43f9-9a33-0cc9bf174961 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:51.653913 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:51.682320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db0fbb9-3b02-4043-97eb-9a5f9735dd1a admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:51.689170 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802a999-80d2-4ceb-ad76-39ce1d9fe792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:51.689340 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802a999-80d2-4ceb-ad76-39ce1d9fe792 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:51.689604 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802a999-80d2-4ceb-ad76-39ce1d9fe792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:51.690579 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802a999-80d2-4ceb-ad76-39ce1d9fe792 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:51.691220 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e802a999-80d2-4ceb-ad76-39ce1d9fe792 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:51.694312 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e802a999-80d2-4ceb-ad76-39ce1d9fe792 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:51.704084 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:51.714011 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5f611-d94b-4fee-a884-13fa8c5918ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:51.714699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5f611-d94b-4fee-a884-13fa8c5918ff admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:51.714984 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5f611-d94b-4fee-a884-13fa8c5918ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:51.715865 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5f611-d94b-4fee-a884-13fa8c5918ff admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:51.716510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a5f611-d94b-4fee-a884-13fa8c5918ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:51.718497 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a5f611-d94b-4fee-a884-13fa8c5918ff admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:51.722032 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 787/1583] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:33:51 2022] POST /identity/v3/users => generated 297 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:51.722934 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802a999-80d2-4ceb-ad76-39ce1d9fe792 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:51.734115 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802a999-80d2-4ceb-ad76-39ce1d9fe792 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:51.740691 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252c870-09fe-4729-88f7-9f6c3ada13ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:51.741551 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252c870-09fe-4729-88f7-9f6c3ada13ee admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:51.741930 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252c870-09fe-4729-88f7-9f6c3ada13ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:51.742329 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252c870-09fe-4729-88f7-9f6c3ada13ee admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:51.743142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2252c870-09fe-4729-88f7-9f6c3ada13ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:51.745310 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2252c870-09fe-4729-88f7-9f6c3ada13ee admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:51.758854 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 797/1584] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:33:51 2022] POST /identity/v3/users => generated 289 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:51.759702 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5f611-d94b-4fee-a884-13fa8c5918ff admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:51.772122 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5f611-d94b-4fee-a884-13fa8c5918ff admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:51.774758 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 788/1585] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:33:51 2022] POST /identity/v3/users => generated 295 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:51.775810 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252c870-09fe-4729-88f7-9f6c3ada13ee admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:51.777644 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740494ac-18c1-45ac-95ba-d8fc50e3b387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:51.778845 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740494ac-18c1-45ac-95ba-d8fc50e3b387 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:51.779119 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740494ac-18c1-45ac-95ba-d8fc50e3b387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:51.779392 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740494ac-18c1-45ac-95ba-d8fc50e3b387 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:51.780036 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740494ac-18c1-45ac-95ba-d8fc50e3b387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:51.782047 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740494ac-18c1-45ac-95ba-d8fc50e3b387 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:51.789167 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252c870-09fe-4729-88f7-9f6c3ada13ee admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:51.799605 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 798/1586] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:51.799605 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740494ac-18c1-45ac-95ba-d8fc50e3b387 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:51.799897 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eea260-1b52-4f1e-93cc-3996aaa53e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:51.799897 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eea260-1b52-4f1e-93cc-3996aaa53e45 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:51.799897 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eea260-1b52-4f1e-93cc-3996aaa53e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:51.799897 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eea260-1b52-4f1e-93cc-3996aaa53e45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:51.800264 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eea260-1b52-4f1e-93cc-3996aaa53e45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:51.801838 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eea260-1b52-4f1e-93cc-3996aaa53e45 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:51.813737 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740494ac-18c1-45ac-95ba-d8fc50e3b387 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:51.818710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9b919-929c-407e-ab19-0e215cfb93c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:51.819611 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9b919-929c-407e-ab19-0e215cfb93c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:51.820219 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9b919-929c-407e-ab19-0e215cfb93c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:51.820509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9b919-929c-407e-ab19-0e215cfb93c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:51.821246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9b919-929c-407e-ab19-0e215cfb93c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:51.823433 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9b919-929c-407e-ab19-0e215cfb93c2 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:51.825933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 789/1587] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:51.826817 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eea260-1b52-4f1e-93cc-3996aaa53e45 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:51.845509 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eea260-1b52-4f1e-93cc-3996aaa53e45 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:51.852741 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b80ec-234f-41bd-8593-194732b005cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:51.853587 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b80ec-234f-41bd-8593-194732b005cc admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:51.854079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b80ec-234f-41bd-8593-194732b005cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:51.854507 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b80ec-234f-41bd-8593-194732b005cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:51.855488 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b80ec-234f-41bd-8593-194732b005cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:51.857934 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b80ec-234f-41bd-8593-194732b005cc admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:51.862307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 799/1588] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:33:51 2022] POST /identity/v3/users => generated 287 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:51.863291 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9b919-929c-407e-ab19-0e215cfb93c2 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:51.872469 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 790/1589] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:51.873387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b80ec-234f-41bd-8593-194732b005cc admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:51.874364 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9b919-929c-407e-ab19-0e215cfb93c2 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:51.879395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e42ab-f06b-4de9-a2cb-a1bd397054be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:51.880118 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e42ab-f06b-4de9-a2cb-a1bd397054be admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:51.880688 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e42ab-f06b-4de9-a2cb-a1bd397054be admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:51.881156 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e42ab-f06b-4de9-a2cb-a1bd397054be admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:51.881967 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e42ab-f06b-4de9-a2cb-a1bd397054be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:51.884001 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e42ab-f06b-4de9-a2cb-a1bd397054be admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:51.896387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b80ec-234f-41bd-8593-194732b005cc admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:51.896387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a8e80-c0cd-44c1-b820-0041dbd1aaae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:51.896387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a8e80-c0cd-44c1-b820-0041dbd1aaae admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:51.896387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a8e80-c0cd-44c1-b820-0041dbd1aaae admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:51.896763 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 800/1590] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:51.896853 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a8e80-c0cd-44c1-b820-0041dbd1aaae admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:51.897049 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e42ab-f06b-4de9-a2cb-a1bd397054be admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:51.903665 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a8e80-c0cd-44c1-b820-0041dbd1aaae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:51.903665 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a8e80-c0cd-44c1-b820-0041dbd1aaae admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:51.909157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e42ab-f06b-4de9-a2cb-a1bd397054be admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:51.915867 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 791/1591] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:51.915867 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a8e80-c0cd-44c1-b820-0041dbd1aaae admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:51.917157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01724c-4e3b-4530-93b5-e77b0455e720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:51.917898 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01724c-4e3b-4530-93b5-e77b0455e720 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:51.918343 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01724c-4e3b-4530-93b5-e77b0455e720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:51.918920 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01724c-4e3b-4530-93b5-e77b0455e720 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:51.919876 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01724c-4e3b-4530-93b5-e77b0455e720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:51.922922 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01724c-4e3b-4530-93b5-e77b0455e720 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:51.924529 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a8e80-c0cd-44c1-b820-0041dbd1aaae admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:51.929275 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd29e05-5c7c-4fd5-a824-dbfdd42bbff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:51.930031 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd29e05-5c7c-4fd5-a824-dbfdd42bbff5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:51.930450 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd29e05-5c7c-4fd5-a824-dbfdd42bbff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:51.930842 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd29e05-5c7c-4fd5-a824-dbfdd42bbff5 admin admin] PATH_INFO: `/v3/projects/ed187feab3f94be38411d79b7600d191/users/d93e5e6e52fd4b98b36aee5b18cfb249/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:51.940745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 801/1592] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:51.940745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01724c-4e3b-4530-93b5-e77b0455e720 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:51.946536 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd29e05-5c7c-4fd5-a824-dbfdd42bbff5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed187feab3f94be38411d79b7600d191, user_id=d93e5e6e52fd4b98b36aee5b18cfb249, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:51.949288 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd29e05-5c7c-4fd5-a824-dbfdd42bbff5 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:51.951284 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01724c-4e3b-4530-93b5-e77b0455e720 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:51.956964 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8993685-50a2-4346-bf3d-45d239cbfe77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:51.957702 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8993685-50a2-4346-bf3d-45d239cbfe77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:51.957988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8993685-50a2-4346-bf3d-45d239cbfe77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:51.958516 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8993685-50a2-4346-bf3d-45d239cbfe77 admin admin] PATH_INFO: `/v3/projects/5c40a340919a49cc8b73687df600e2ea/users/b172f56cbd604342ba6d8169c969e157/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:51.965048 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 792/1593] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:51 2022] PUT /identity/v3/projects/ed187feab3f94be38411d79b7600d191/users/d93e5e6e52fd4b98b36aee5b18cfb249/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:51.966170 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd29e05-5c7c-4fd5-a824-dbfdd42bbff5 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:51.981421 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8993685-50a2-4346-bf3d-45d239cbfe77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c40a340919a49cc8b73687df600e2ea, user_id=b172f56cbd604342ba6d8169c969e157, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:51.984236 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8993685-50a2-4346-bf3d-45d239cbfe77 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:51.999238 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 802/1594] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:51 2022] PUT /identity/v3/projects/5c40a340919a49cc8b73687df600e2ea/users/b172f56cbd604342ba6d8169c969e157/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:52.000051 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8993685-50a2-4346-bf3d-45d239cbfe77 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:52.037289 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd29e05-5c7c-4fd5-a824-dbfdd42bbff5 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:52.067042 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8993685-50a2-4346-bf3d-45d239cbfe77 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:52.072224 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be08e3a-a821-4c21-990f-d3bebd76ad4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:52.072830 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be08e3a-a821-4c21-990f-d3bebd76ad4e admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:52.073109 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be08e3a-a821-4c21-990f-d3bebd76ad4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:52.073382 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be08e3a-a821-4c21-990f-d3bebd76ad4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:52.074226 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be08e3a-a821-4c21-990f-d3bebd76ad4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:52.075864 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be08e3a-a821-4c21-990f-d3bebd76ad4e admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:52.085928 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 803/1595] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:52.086757 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be08e3a-a821-4c21-990f-d3bebd76ad4e admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:52.095660 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa97b0-e540-42bd-957e-17fad7c54371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:52.096202 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa97b0-e540-42bd-957e-17fad7c54371 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:52.096202 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa97b0-e540-42bd-957e-17fad7c54371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:52.096410 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa97b0-e540-42bd-957e-17fad7c54371 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:52.096672 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be08e3a-a821-4c21-990f-d3bebd76ad4e admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:52.097013 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa97b0-e540-42bd-957e-17fad7c54371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:52.099241 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa97b0-e540-42bd-957e-17fad7c54371 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:52.102851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c15dad1-6036-4446-aecf-e3f5dd1795f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:52.103451 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15dad1-6036-4446-aecf-e3f5dd1795f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:52.103732 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15dad1-6036-4446-aecf-e3f5dd1795f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:52.104014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15dad1-6036-4446-aecf-e3f5dd1795f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:52.104642 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c15dad1-6036-4446-aecf-e3f5dd1795f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:52.107149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c15dad1-6036-4446-aecf-e3f5dd1795f2 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:52.110317 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 793/1596] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:52.111137 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa97b0-e540-42bd-957e-17fad7c54371 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:52.121346 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 804/1597] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:52.122105 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c15dad1-6036-4446-aecf-e3f5dd1795f2 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:52.128834 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa97b0-e540-42bd-957e-17fad7c54371 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:52.132423 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c15dad1-6036-4446-aecf-e3f5dd1795f2 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:52.135427 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d71e12-7f74-4540-9284-75c464e1e9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:52.136087 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d71e12-7f74-4540-9284-75c464e1e9fa admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:52.136326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d71e12-7f74-4540-9284-75c464e1e9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:52.136603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d71e12-7f74-4540-9284-75c464e1e9fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:52.137284 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d71e12-7f74-4540-9284-75c464e1e9fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:52.138751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f794e-c3ac-498a-a2aa-86284be9b598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:52.138918 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d71e12-7f74-4540-9284-75c464e1e9fa admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:52.139371 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f794e-c3ac-498a-a2aa-86284be9b598 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:52.139651 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f794e-c3ac-498a-a2aa-86284be9b598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:52.139920 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f794e-c3ac-498a-a2aa-86284be9b598 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:52.140722 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f794e-c3ac-498a-a2aa-86284be9b598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:52.142378 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f794e-c3ac-498a-a2aa-86284be9b598 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:52.152122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 794/1598] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:52.152122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415492a4-fe38-4194-a3de-1187648d050f None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:52.152122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415492a4-fe38-4194-a3de-1187648d050f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:52.152122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415492a4-fe38-4194-a3de-1187648d050f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:52.156693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 805/1599] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:52.156693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa4ab6-0e36-4c20-abb8-1652493d740d None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:52.156693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa4ab6-0e36-4c20-abb8-1652493d740d None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:52.156693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa4ab6-0e36-4c20-abb8-1652493d740d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:52.165197 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:52.172928 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:52.179424 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-59aa4ab6-0e36-4c20-abb8-1652493d740d None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:52.180894 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-59aa4ab6-0e36-4c20-abb8-1652493d740d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:52.183070 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-415492a4-fe38-4194-a3de-1187648d050f None None] MFA Rules not processed for user `d93e5e6e52fd4b98b36aee5b18cfb249`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:52.184824 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-415492a4-fe38-4194-a3de-1187648d050f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:52.197510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 806/1600] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:52.197510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aa4ab6-0e36-4c20-abb8-1652493d740d None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:52.206694 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aa4ab6-0e36-4c20-abb8-1652493d740d None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:52.211487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7353a6-b8dd-4548-a5f4-a31a4486c98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:52.212135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7353a6-b8dd-4548-a5f4-a31a4486c98d admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:52.212412 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7353a6-b8dd-4548-a5f4-a31a4486c98d admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:52.212808 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7353a6-b8dd-4548-a5f4-a31a4486c98d admin admin] PATH_INFO: `/v3/projects/170a066ef9584da0915a40ff4379c985/users/d5e7228941074b579d0b11334226169c/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:52.228294 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7353a6-b8dd-4548-a5f4-a31a4486c98d admin admin] RBAC: Authorizing `identity:create_grant(project_id=170a066ef9584da0915a40ff4379c985, user_id=d5e7228941074b579d0b11334226169c, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:52.231542 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7353a6-b8dd-4548-a5f4-a31a4486c98d admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:52.245785 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 807/1601] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:52 2022] PUT /identity/v3/projects/170a066ef9584da0915a40ff4379c985/users/d5e7228941074b579d0b11334226169c/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:52.246720 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7353a6-b8dd-4548-a5f4-a31a4486c98d admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:52.286064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 795/1602] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:33:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:52.286064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415492a4-fe38-4194-a3de-1187648d050f None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:52.301643 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415492a4-fe38-4194-a3de-1187648d050f None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:52.350598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7353a6-b8dd-4548-a5f4-a31a4486c98d admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:52.353560 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f0130-c2da-44ee-94de-70bc4390f523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:52.354315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f0130-c2da-44ee-94de-70bc4390f523 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:52.355061 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f0130-c2da-44ee-94de-70bc4390f523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:52.355423 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f0130-c2da-44ee-94de-70bc4390f523 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:52.356436 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f0130-c2da-44ee-94de-70bc4390f523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:52.358920 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f0130-c2da-44ee-94de-70bc4390f523 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:52.370125 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 808/1603] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:52.371143 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f0130-c2da-44ee-94de-70bc4390f523 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:52.374848 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a243a-2593-4dea-93dd-b071057bfbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:52.375573 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a243a-2593-4dea-93dd-b071057bfbb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:52.376021 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a243a-2593-4dea-93dd-b071057bfbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:52.376638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a243a-2593-4dea-93dd-b071057bfbb0 admin admin] PATH_INFO: `/v3/projects/b4bbb5d3895c49fa8695d3bca946dbea/users/8020db56bca04d61b5a6e40384c25715/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:52.382569 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f0130-c2da-44ee-94de-70bc4390f523 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:52.387788 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205fafcb-6e3f-42d5-8ec2-e7637e411f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:52.388469 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fafcb-6e3f-42d5-8ec2-e7637e411f5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:52.388754 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fafcb-6e3f-42d5-8ec2-e7637e411f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:52.389019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fafcb-6e3f-42d5-8ec2-e7637e411f5c admin admin] PATH_INFO: `/v3/projects/aab96a55de014d8cac2c004ba7cadedd/users/73624e79ccdd4782beddf663bb6d02e1/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:52.392880 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474a243a-2593-4dea-93dd-b071057bfbb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4bbb5d3895c49fa8695d3bca946dbea, user_id=8020db56bca04d61b5a6e40384c25715, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:52.395646 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474a243a-2593-4dea-93dd-b071057bfbb0 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:52.406009 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205fafcb-6e3f-42d5-8ec2-e7637e411f5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=aab96a55de014d8cac2c004ba7cadedd, user_id=73624e79ccdd4782beddf663bb6d02e1, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:52.410037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205fafcb-6e3f-42d5-8ec2-e7637e411f5c admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:52.419990 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 796/1604] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:52 2022] PUT /identity/v3/projects/b4bbb5d3895c49fa8695d3bca946dbea/users/8020db56bca04d61b5a6e40384c25715/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:52.420748 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a243a-2593-4dea-93dd-b071057bfbb0 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:52.430473 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 809/1605] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:52 2022] PUT /identity/v3/projects/aab96a55de014d8cac2c004ba7cadedd/users/73624e79ccdd4782beddf663bb6d02e1/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:52.430473 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205fafcb-6e3f-42d5-8ec2-e7637e411f5c admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:52.491348 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a243a-2593-4dea-93dd-b071057bfbb0 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:52.491790 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205fafcb-6e3f-42d5-8ec2-e7637e411f5c admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:52.496023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c1de1-1b07-4541-8adf-d0060a4262fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:52.496784 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c1de1-1b07-4541-8adf-d0060a4262fe service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:52.497237 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c1de1-1b07-4541-8adf-d0060a4262fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:52.497613 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c1de1-1b07-4541-8adf-d0060a4262fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:52.519723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:52.557922 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a374c1b-4fb2-49a2-89cd-9be5e593aef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:52.558624 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a374c1b-4fb2-49a2-89cd-9be5e593aef1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:52.559429 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a374c1b-4fb2-49a2-89cd-9be5e593aef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:52.559708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a374c1b-4fb2-49a2-89cd-9be5e593aef1 admin admin] PATH_INFO: `/v3/projects/cfb0b630e22745e8af15fb02c71800c3/users/8388303f9098497bb559b3bcb59fab52/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:52.573063 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165c1de1-1b07-4541-8adf-d0060a4262fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:52.573993 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a374c1b-4fb2-49a2-89cd-9be5e593aef1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfb0b630e22745e8af15fb02c71800c3, user_id=8388303f9098497bb559b3bcb59fab52, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:52.574605 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165c1de1-1b07-4541-8adf-d0060a4262fe service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:52.577391 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a374c1b-4fb2-49a2-89cd-9be5e593aef1 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:52.579061 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:52.586609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 810/1606] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:52.588891 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe742b3-acbf-46fc-8e65-acedd16b1ef5 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:52.589717 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe742b3-acbf-46fc-8e65-acedd16b1ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:52.590625 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe742b3-acbf-46fc-8e65-acedd16b1ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:52.592740 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 797/1607] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:52 2022] PUT /identity/v3/projects/cfb0b630e22745e8af15fb02c71800c3/users/8388303f9098497bb559b3bcb59fab52/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:52.593350 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a374c1b-4fb2-49a2-89cd-9be5e593aef1 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:52.623803 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-dfe742b3-acbf-46fc-8e65-acedd16b1ef5 None None] MFA Rules not processed for user `d5e7228941074b579d0b11334226169c`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:52.625028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-dfe742b3-acbf-46fc-8e65-acedd16b1ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:52.670327 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a374c1b-4fb2-49a2-89cd-9be5e593aef1 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:52.678363 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105cba0-8d7c-4059-8345-c931d0235293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:52.679142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105cba0-8d7c-4059-8345-c931d0235293 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:52.679466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105cba0-8d7c-4059-8345-c931d0235293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:52.679780 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105cba0-8d7c-4059-8345-c931d0235293 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:52.680448 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105cba0-8d7c-4059-8345-c931d0235293 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:52.682574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105cba0-8d7c-4059-8345-c931d0235293 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:52.710551 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 811/1608] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:33:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:52.712047 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe742b3-acbf-46fc-8e65-acedd16b1ef5 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:52.715708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 798/1609] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:33:52 2022] POST /identity/v3/projects => generated 379 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:52.716038 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894145a-76d8-432a-8ea5-ab7e796e01fe None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:52.716336 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894145a-76d8-432a-8ea5-ab7e796e01fe None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:52.716639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894145a-76d8-432a-8ea5-ab7e796e01fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:52.732364 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe742b3-acbf-46fc-8e65-acedd16b1ef5 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:52.739728 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-d894145a-76d8-432a-8ea5-ab7e796e01fe None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:52.741150 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d894145a-76d8-432a-8ea5-ab7e796e01fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:52.823584 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0c980-93fe-4c3c-8540-576160e5cfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:52.824348 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0c980-93fe-4c3c-8540-576160e5cfe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:52.825345 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0c980-93fe-4c3c-8540-576160e5cfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:52.825484 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 799/1610] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:52.829449 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0c980-93fe-4c3c-8540-576160e5cfe8 admin admin] PATH_INFO: `/v3/projects/66f2b3d1b1dc464f8e0501519f3b211e/users/ec2654278e1f4717aa7f49f3f9c73504/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:52.830339 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03149c-9c07-406c-9833-91c9aea2ddb7 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:52.830339 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03149c-9c07-406c-9833-91c9aea2ddb7 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:52.830339 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03149c-9c07-406c-9833-91c9aea2ddb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:52.849585 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-8b03149c-9c07-406c-9833-91c9aea2ddb7 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:52.850997 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8b03149c-9c07-406c-9833-91c9aea2ddb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:52.853138 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0c980-93fe-4c3c-8540-576160e5cfe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66f2b3d1b1dc464f8e0501519f3b211e, user_id=ec2654278e1f4717aa7f49f3f9c73504, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:52.856625 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0c980-93fe-4c3c-8540-576160e5cfe8 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:52.860754 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 800/1611] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:52.862532 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bacf8-722d-4f1a-93d6-d43ae94261ea None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:52.862791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bacf8-722d-4f1a-93d6-d43ae94261ea None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:52.863101 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bacf8-722d-4f1a-93d6-d43ae94261ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:52.880855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 812/1612] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:52 2022] PUT /identity/v3/projects/66f2b3d1b1dc464f8e0501519f3b211e/users/ec2654278e1f4717aa7f49f3f9c73504/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:52.882051 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0c980-93fe-4c3c-8540-576160e5cfe8 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:52.885681 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-237bacf8-722d-4f1a-93d6-d43ae94261ea None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:52.887058 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-237bacf8-722d-4f1a-93d6-d43ae94261ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:52.934524 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:53.007289 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 801/1613] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:53.009040 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2925ec-fb01-4f3a-a7df-48cc1d922af6 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:53.009570 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2925ec-fb01-4f3a-a7df-48cc1d922af6 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:53.012218 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2925ec-fb01-4f3a-a7df-48cc1d922af6 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:53.013885 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0c980-93fe-4c3c-8540-576160e5cfe8 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:53.015257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 802/1614] 10.222.0.18 () {54 vars in 962 bytes} [Thu Sep 1 16:33:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:33:53.016668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2925ec-fb01-4f3a-a7df-48cc1d922af6 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:53.020311 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8e2925ec-fb01-4f3a-a7df-48cc1d922af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:53.024360 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521007a-34c4-48f4-a264-55eb0d49bc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:53.025068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521007a-34c4-48f4-a264-55eb0d49bc82 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:53.025922 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521007a-34c4-48f4-a264-55eb0d49bc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:53.027102 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521007a-34c4-48f4-a264-55eb0d49bc82 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:53.027908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d521007a-34c4-48f4-a264-55eb0d49bc82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:53.030365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d521007a-34c4-48f4-a264-55eb0d49bc82 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:53.072792 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:53.079501 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 813/1615] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:33:52 2022] POST /identity/v3/users => generated 296 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:53.080405 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521007a-34c4-48f4-a264-55eb0d49bc82 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:53.084939 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-d521007a-34c4-48f4-a264-55eb0d49bc82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:53.118488 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2925ec-fb01-4f3a-a7df-48cc1d922af6 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:53.123778 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894cfff-70b6-439a-8174-266ef1870275 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:53.124635 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894cfff-70b6-439a-8174-266ef1870275 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:53.125061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894cfff-70b6-439a-8174-266ef1870275 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:53.125482 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894cfff-70b6-439a-8174-266ef1870275 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:53.136975 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5894cfff-70b6-439a-8174-266ef1870275 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:53.139177 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5894cfff-70b6-439a-8174-266ef1870275 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:53.152294 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 803/1616] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:53.153856 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521007a-34c4-48f4-a264-55eb0d49bc82 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:53.154177 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e22df3-8792-4840-8154-7910936f3d37 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:53.154481 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e22df3-8792-4840-8154-7910936f3d37 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:53.154865 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e22df3-8792-4840-8154-7910936f3d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:53.159636 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea79454-b7db-4992-b72f-3b26a2116e86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:53.160531 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea79454-b7db-4992-b72f-3b26a2116e86 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:53.160810 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea79454-b7db-4992-b72f-3b26a2116e86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:53.161161 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea79454-b7db-4992-b72f-3b26a2116e86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:53.178498 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea79454-b7db-4992-b72f-3b26a2116e86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:53.180459 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea79454-b7db-4992-b72f-3b26a2116e86 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:53.193245 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-b6e22df3-8792-4840-8154-7910936f3d37 None None] MFA Rules not processed for user `ec2654278e1f4717aa7f49f3f9c73504`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:53.194662 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b6e22df3-8792-4840-8154-7910936f3d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:53.194871 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 814/1617] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:53.195831 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea79454-b7db-4992-b72f-3b26a2116e86 service neutron] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:53.206910 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea79454-b7db-4992-b72f-3b26a2116e86 service neutron] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:53.213643 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6cc541-93c5-4eee-ae60-f5b34c43bff4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:53.214798 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6cc541-93c5-4eee-ae60-f5b34c43bff4 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:53.215229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6cc541-93c5-4eee-ae60-f5b34c43bff4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:53.215708 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6cc541-93c5-4eee-ae60-f5b34c43bff4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:53.235125 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6cc541-93c5-4eee-ae60-f5b34c43bff4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:53.240382 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6cc541-93c5-4eee-ae60-f5b34c43bff4 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:53.265133 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 815/1618] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:53.266857 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f1e20-c2d5-4c87-8a75-a5483b5f0932 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:53.267251 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f1e20-c2d5-4c87-8a75-a5483b5f0932 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:53.267633 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f1e20-c2d5-4c87-8a75-a5483b5f0932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:53.325129 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-937f1e20-c2d5-4c87-8a75-a5483b5f0932 None None] MFA Rules not processed for user `278e6d6aed414c758fe3e9f4df6bf81e`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:53.326942 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-937f1e20-c2d5-4c87-8a75-a5483b5f0932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:53.332101 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 804/1619] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:33:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:53.333838 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e22df3-8792-4840-8154-7910936f3d37 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:53.362193 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e22df3-8792-4840-8154-7910936f3d37 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:53.373081 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514bae1c-31cf-4f66-a6bf-f3839fc9bfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:53.373769 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bae1c-31cf-4f66-a6bf-f3839fc9bfb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:53.374057 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bae1c-31cf-4f66-a6bf-f3839fc9bfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:53.374374 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bae1c-31cf-4f66-a6bf-f3839fc9bfb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:53.375144 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514bae1c-31cf-4f66-a6bf-f3839fc9bfb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:53.376688 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514bae1c-31cf-4f66-a6bf-f3839fc9bfb8 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:53.394990 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 805/1620] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:53.396658 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f467e3d-3023-4748-897d-6c7f0802e9dd None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:53.396952 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f467e3d-3023-4748-897d-6c7f0802e9dd None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:53.397550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f467e3d-3023-4748-897d-6c7f0802e9dd None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:53.405252 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 806/1621] 10.222.0.18 () {54 vars in 962 bytes} [Thu Sep 1 16:33:53 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:33:53.405986 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f467e3d-3023-4748-897d-6c7f0802e9dd None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:53.425059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f467e3d-3023-4748-897d-6c7f0802e9dd None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:53.433050 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f9697-3e59-4eb8-af31-9a4fc65940c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:53.434098 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f9697-3e59-4eb8-af31-9a4fc65940c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:53.434418 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f9697-3e59-4eb8-af31-9a4fc65940c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:53.434696 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f9697-3e59-4eb8-af31-9a4fc65940c4 admin admin] PATH_INFO: `/v3/projects/15bc0930beb341cfb9f5e8293c274efe/users/8e7d5f3113c54e328e02a25d7af63964/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:53.458897 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f9697-3e59-4eb8-af31-9a4fc65940c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15bc0930beb341cfb9f5e8293c274efe, user_id=8e7d5f3113c54e328e02a25d7af63964, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:53.463426 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f9697-3e59-4eb8-af31-9a4fc65940c4 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:53.487892 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 807/1622] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:53 2022] PUT /identity/v3/projects/15bc0930beb341cfb9f5e8293c274efe/users/8e7d5f3113c54e328e02a25d7af63964/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:53.491070 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf594cf3-f156-4005-9097-12889cce1109 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:53.491364 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf594cf3-f156-4005-9097-12889cce1109 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:53.491847 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf594cf3-f156-4005-9097-12889cce1109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:53.511613 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 816/1623] 10.222.0.18 () {58 vars in 1059 bytes} [Thu Sep 1 16:33:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:53.512713 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937f1e20-c2d5-4c87-8a75-a5483b5f0932 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:53.533656 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-cf594cf3-f156-4005-9097-12889cce1109 None None] MFA Rules not processed for user `278e6d6aed414c758fe3e9f4df6bf81e`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:53.535680 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-cf594cf3-f156-4005-9097-12889cce1109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:53.538140 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937f1e20-c2d5-4c87-8a75-a5483b5f0932 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:53.584849 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:53.673443 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60cfc4-0fce-4516-a152-c736f43e0fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:53.674197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60cfc4-0fce-4516-a152-c736f43e0fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:53.674680 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60cfc4-0fce-4516-a152-c736f43e0fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:53.674980 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60cfc4-0fce-4516-a152-c736f43e0fc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:53.675737 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f60cfc4-0fce-4516-a152-c736f43e0fc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:53.678757 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f60cfc4-0fce-4516-a152-c736f43e0fc3 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:53.680621 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 808/1624] 10.222.0.18 () {58 vars in 1059 bytes} [Thu Sep 1 16:33:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:53.681503 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf594cf3-f156-4005-9097-12889cce1109 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:53.700142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 817/1625] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:53.701569 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34171b28-743f-4027-9e3a-c37cdf944c71 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:53.701958 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34171b28-743f-4027-9e3a-c37cdf944c71 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:53.702406 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34171b28-743f-4027-9e3a-c37cdf944c71 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:53.703352 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf594cf3-f156-4005-9097-12889cce1109 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:53.707635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 818/1626] 10.222.0.18 () {54 vars in 912 bytes} [Thu Sep 1 16:33:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:33:53.708556 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34171b28-743f-4027-9e3a-c37cdf944c71 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:53.714595 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb23ae85-abcb-4057-b405-8edd40e0c646 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:53.715903 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23ae85-abcb-4057-b405-8edd40e0c646 service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:53.716339 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23ae85-abcb-4057-b405-8edd40e0c646 service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:53.716512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23ae85-abcb-4057-b405-8edd40e0c646 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:53.727959 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:53.740645 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34171b28-743f-4027-9e3a-c37cdf944c71 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:53.746567 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752c906-3050-4778-a1e3-e5b40b8a2f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:53.746782 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752c906-3050-4778-a1e3-e5b40b8a2f69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:53.747108 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752c906-3050-4778-a1e3-e5b40b8a2f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:53.747246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752c906-3050-4778-a1e3-e5b40b8a2f69 admin admin] PATH_INFO: `/v3/projects/15bc0930beb341cfb9f5e8293c274efe/users/8e7d5f3113c54e328e02a25d7af63964/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:53.752116 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f752c906-3050-4778-a1e3-e5b40b8a2f69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15bc0930beb341cfb9f5e8293c274efe, user_id=8e7d5f3113c54e328e02a25d7af63964, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:53.760173 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f752c906-3050-4778-a1e3-e5b40b8a2f69 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:53.797339 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 819/1627] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:53 2022] PUT /identity/v3/projects/15bc0930beb341cfb9f5e8293c274efe/users/8e7d5f3113c54e328e02a25d7af63964/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:53.800193 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61198319-8197-469b-9140-5abb6f69b0b8 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:53.800497 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61198319-8197-469b-9140-5abb6f69b0b8 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:53.801819 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61198319-8197-469b-9140-5abb6f69b0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:53.844737 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-61198319-8197-469b-9140-5abb6f69b0b8 None None] MFA Rules not processed for user `278e6d6aed414c758fe3e9f4df6bf81e`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:53.845184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb23ae85-abcb-4057-b405-8edd40e0c646 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:53.846077 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-61198319-8197-469b-9140-5abb6f69b0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:53.846925 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb23ae85-abcb-4057-b405-8edd40e0c646 service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:53.875132 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 809/1628] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:53.877680 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8c9e3-b639-4f9e-96b5-980a564a1208 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:53.877974 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8c9e3-b639-4f9e-96b5-980a564a1208 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:53.878264 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8c9e3-b639-4f9e-96b5-980a564a1208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:53.971082 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-d4b8c9e3-b639-4f9e-96b5-980a564a1208 None None] MFA Rules not processed for user `8e7d5f3113c54e328e02a25d7af63964`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:53.973581 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d4b8c9e3-b639-4f9e-96b5-980a564a1208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:54.008440 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 820/1629] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Sep 1 16:33:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:54.016444 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61198319-8197-469b-9140-5abb6f69b0b8 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:54.142441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61198319-8197-469b-9140-5abb6f69b0b8 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:54.154511 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 810/1630] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:33:53 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:54.158693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3d1af-e318-4018-be56-64f0c6eb256d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:54.160757 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3d1af-e318-4018-be56-64f0c6eb256d service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:54.162730 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3d1af-e318-4018-be56-64f0c6eb256d service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:54.163020 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3d1af-e318-4018-be56-64f0c6eb256d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:54.170860 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b8c9e3-b639-4f9e-96b5-980a564a1208 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:54.186471 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b8c9e3-b639-4f9e-96b5-980a564a1208 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:54.191311 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3d1af-e318-4018-be56-64f0c6eb256d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:54.192356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6b5bb-3d30-4bc4-97dc-75a698504662 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:54.193075 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3d1af-e318-4018-be56-64f0c6eb256d service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:54.193255 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6b5bb-3d30-4bc4-97dc-75a698504662 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:54.193694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6b5bb-3d30-4bc4-97dc-75a698504662 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:54.194075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6b5bb-3d30-4bc4-97dc-75a698504662 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:54.219940 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c6b5bb-3d30-4bc4-97dc-75a698504662 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:54.222993 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 821/1631] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:54.223284 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c6b5bb-3d30-4bc4-97dc-75a698504662 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:54.244673 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 811/1632] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:54 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:33:54.329171 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3d1af-e318-4018-be56-64f0c6eb256d service cinder] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:54.345146 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3d1af-e318-4018-be56-64f0c6eb256d service cinder] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:54.363866 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75cfde-1962-410a-957f-c974d668e7bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:54.364493 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75cfde-1962-410a-957f-c974d668e7bb service nova] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:54.364782 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75cfde-1962-410a-957f-c974d668e7bb service nova] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:54.365069 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75cfde-1962-410a-957f-c974d668e7bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:54.397153 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d40aa-2238-4d0e-9968-d82a385aab80 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:54.397982 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d40aa-2238-4d0e-9968-d82a385aab80 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:54.398359 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d40aa-2238-4d0e-9968-d82a385aab80 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:54.399910 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 812/1633] 10.222.0.18 () {54 vars in 912 bytes} [Thu Sep 1 16:33:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:33:54.411480 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49611c-edcd-4680-8044-24134f8b8566 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:54.411823 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49611c-edcd-4680-8044-24134f8b8566 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:54.412125 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49611c-edcd-4680-8044-24134f8b8566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:54.440697 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-6d49611c-edcd-4680-8044-24134f8b8566 None None] MFA Rules not processed for user `278e6d6aed414c758fe3e9f4df6bf81e`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:54.442010 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6d49611c-edcd-4680-8044-24134f8b8566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:54.453275 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 813/1634] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Sep 1 16:33:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:54.454189 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49611c-edcd-4680-8044-24134f8b8566 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:54.467516 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49611c-edcd-4680-8044-24134f8b8566 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:54.477600 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80785674-8712-4f40-bff0-4264f6ad85a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:54.479389 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80785674-8712-4f40-bff0-4264f6ad85a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:54.479907 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80785674-8712-4f40-bff0-4264f6ad85a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:54.480234 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80785674-8712-4f40-bff0-4264f6ad85a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:54.508210 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75cfde-1962-410a-957f-c974d668e7bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:54.517785 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75cfde-1962-410a-957f-c974d668e7bb service nova] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:54.525564 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:54.540184 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 822/1635] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:54.624968 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80785674-8712-4f40-bff0-4264f6ad85a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:54.633363 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80785674-8712-4f40-bff0-4264f6ad85a7 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:54.654110 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 814/1636] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:54.927292 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75cfde-1962-410a-957f-c974d668e7bb service nova] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:54.951546 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75cfde-1962-410a-957f-c974d668e7bb service nova] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:54.965555 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae0e7f-2027-4f13-9d13-433acdae0fa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:54.967241 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae0e7f-2027-4f13-9d13-433acdae0fa6 service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:54.967241 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae0e7f-2027-4f13-9d13-433acdae0fa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:54.968411 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae0e7f-2027-4f13-9d13-433acdae0fa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:54.996132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae0e7f-2027-4f13-9d13-433acdae0fa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:54.998998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae0e7f-2027-4f13-9d13-433acdae0fa6 service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:55.035326 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 823/1637] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:58.544264 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80785674-8712-4f40-bff0-4264f6ad85a7 service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:58.703248 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80785674-8712-4f40-bff0-4264f6ad85a7 service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:58.717578 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d1abc-be75-45f4-a8d1-4b115956c44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:58.718459 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d1abc-be75-45f4-a8d1-4b115956c44e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:58.718888 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d1abc-be75-45f4-a8d1-4b115956c44e admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:58.719226 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d1abc-be75-45f4-a8d1-4b115956c44e admin admin] PATH_INFO: `/v3/users/ec2654278e1f4717aa7f49f3f9c73504` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:58.721916 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d1abc-be75-45f4-a8d1-4b115956c44e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec2654278e1f4717aa7f49f3f9c73504)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:58.728186 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d1abc-be75-45f4-a8d1-4b115956c44e admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:58.905080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c22d1abc-be75-45f4-a8d1-4b115956c44e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec2654278e1f4717aa7f49f3f9c73504'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:58.921364 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c22d1abc-be75-45f4-a8d1-4b115956c44e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec2654278e1f4717aa7f49f3f9c73504'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:58.922122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c22d1abc-be75-45f4-a8d1-4b115956c44e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec2654278e1f4717aa7f49f3f9c73504'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:58.945010 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c22d1abc-be75-45f4-a8d1-4b115956c44e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec2654278e1f4717aa7f49f3f9c73504'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:58.945937 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c22d1abc-be75-45f4-a8d1-4b115956c44e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec2654278e1f4717aa7f49f3f9c73504'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:58.984037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 815/1638] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:33:58 2022] DELETE /identity/v3/users/ec2654278e1f4717aa7f49f3f9c73504 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:58.988484 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcf06a-1c08-4cd5-9c00-cc3086a2e0ed None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:58.989040 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcf06a-1c08-4cd5-9c00-cc3086a2e0ed None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:58.989249 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcf06a-1c08-4cd5-9c00-cc3086a2e0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:59.032629 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-36fcf06a-1c08-4cd5-9c00-cc3086a2e0ed None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:59.040387 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-36fcf06a-1c08-4cd5-9c00-cc3086a2e0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:59.049364 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:59.137593 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 824/1639] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:33:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:59.153533 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d1abc-be75-45f4-a8d1-4b115956c44e admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:59.155324 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c22d1abc-be75-45f4-a8d1-4b115956c44e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:59.239935 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:59.270124 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d1abc-be75-45f4-a8d1-4b115956c44e admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:59.282995 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7457a-648a-4404-b1c6-85d4128b0d25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:59.285221 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7457a-648a-4404-b1c6-85d4128b0d25 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:59.287743 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7457a-648a-4404-b1c6-85d4128b0d25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:59.288175 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7457a-648a-4404-b1c6-85d4128b0d25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:59.316762 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7457a-648a-4404-b1c6-85d4128b0d25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:59.319742 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7457a-648a-4404-b1c6-85d4128b0d25 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:59.351038 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 816/1640] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:01.889695 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fcf06a-1c08-4cd5-9c00-cc3086a2e0ed None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:01.891346 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-36fcf06a-1c08-4cd5-9c00-cc3086a2e0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:01.909834 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fcf06a-1c08-4cd5-9c00-cc3086a2e0ed None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:01.933119 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3297f-89ef-426b-ba95-cd94289e210d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:01.935835 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3297f-89ef-426b-ba95-cd94289e210d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:01.936370 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3297f-89ef-426b-ba95-cd94289e210d admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:01.936899 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3297f-89ef-426b-ba95-cd94289e210d admin admin] PATH_INFO: `/v3/projects/66f2b3d1b1dc464f8e0501519f3b211e` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:01.939686 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b3297f-89ef-426b-ba95-cd94289e210d admin admin] RBAC: Authorizing `identity:delete_project(project_id=66f2b3d1b1dc464f8e0501519f3b211e)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:01.942154 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b3297f-89ef-426b-ba95-cd94289e210d admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:02.071619 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-55b3297f-89ef-426b-ba95-cd94289e210d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66f2b3d1b1dc464f8e0501519f3b211e', 'request_id': 'req-55b3297f-89ef-426b-ba95-cd94289e210d'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:02.087872 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-55b3297f-89ef-426b-ba95-cd94289e210d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66f2b3d1b1dc464f8e0501519f3b211e', 'request_id': 'req-55b3297f-89ef-426b-ba95-cd94289e210d'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:02.114262 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-55b3297f-89ef-426b-ba95-cd94289e210d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66f2b3d1b1dc464f8e0501519f3b211e', 'request_id': 'req-55b3297f-89ef-426b-ba95-cd94289e210d'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:02.162967 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-55b3297f-89ef-426b-ba95-cd94289e210d admin admin] The token cache is being invalidate because project 66f2b3d1b1dc464f8e0501519f3b211e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:02.163446 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-55b3297f-89ef-426b-ba95-cd94289e210d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:02.164194 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-55b3297f-89ef-426b-ba95-cd94289e210d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:02.173722 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 825/1641] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:01 2022] DELETE /identity/v3/projects/66f2b3d1b1dc464f8e0501519f3b211e => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:02.178569 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195d64b-d4b6-4b92-8b02-ddc7342624f5 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:02.179926 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195d64b-d4b6-4b92-8b02-ddc7342624f5 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:02.180400 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195d64b-d4b6-4b92-8b02-ddc7342624f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:02.216151 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-0195d64b-d4b6-4b92-8b02-ddc7342624f5 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:02.218055 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0195d64b-d4b6-4b92-8b02-ddc7342624f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:02.387862 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 817/1642] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:02.396048 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d52dd1-4eff-4270-a0b1-da5fe0c50999 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:02.396294 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d52dd1-4eff-4270-a0b1-da5fe0c50999 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:02.398731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d52dd1-4eff-4270-a0b1-da5fe0c50999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:02.445337 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-22d52dd1-4eff-4270-a0b1-da5fe0c50999 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:02.448550 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-22d52dd1-4eff-4270-a0b1-da5fe0c50999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:02.481247 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 826/1643] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:02.484487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195d64b-d4b6-4b92-8b02-ddc7342624f5 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:02.498285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195d64b-d4b6-4b92-8b02-ddc7342624f5 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:02.515739 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c69c06-13f2-419f-9062-96ad4872f3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:02.516469 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c69c06-13f2-419f-9062-96ad4872f3d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:02.516768 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c69c06-13f2-419f-9062-96ad4872f3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:02.517051 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c69c06-13f2-419f-9062-96ad4872f3d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:02.517725 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c69c06-13f2-419f-9062-96ad4872f3d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:34:02.518035 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c69c06-13f2-419f-9062-96ad4872f3d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:02.520254 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c69c06-13f2-419f-9062-96ad4872f3d1 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:02.550710 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 818/1644] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:34:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:34:02.553243 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d52dd1-4eff-4270-a0b1-da5fe0c50999 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:02.568814 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d52dd1-4eff-4270-a0b1-da5fe0c50999 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:02.584344 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ee0f9-de2a-4f9e-8c25-f0264f895c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:02.585050 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ee0f9-de2a-4f9e-8c25-f0264f895c51 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:02.585335 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ee0f9-de2a-4f9e-8c25-f0264f895c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:02.585636 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ee0f9-de2a-4f9e-8c25-f0264f895c51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:02.586338 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26ee0f9-de2a-4f9e-8c25-f0264f895c51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:02.588322 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26ee0f9-de2a-4f9e-8c25-f0264f895c51 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:02.674676 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c69c06-13f2-419f-9062-96ad4872f3d1 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:02.678635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 827/1645] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:02 2022] POST /identity/v3/projects => generated 375 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:02.698958 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c69c06-13f2-419f-9062-96ad4872f3d1 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:02.851080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d52d8c8-d712-4777-acfc-a055cebb211e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:02.851708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52d8c8-d712-4777-acfc-a055cebb211e admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:02.851989 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52d8c8-d712-4777-acfc-a055cebb211e admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:02.852258 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52d8c8-d712-4777-acfc-a055cebb211e admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:02.852967 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d52d8c8-d712-4777-acfc-a055cebb211e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:02.855816 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d52d8c8-d712-4777-acfc-a055cebb211e admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:02.905829 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 819/1646] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:02 2022] POST /identity/v3/users => generated 288 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:34:02.909684 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ee0f9-de2a-4f9e-8c25-f0264f895c51 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:02.937024 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ee0f9-de2a-4f9e-8c25-f0264f895c51 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:02.944828 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33048eee-d623-464f-8987-9219ac4257fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:02.946069 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33048eee-d623-464f-8987-9219ac4257fd admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:02.946566 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33048eee-d623-464f-8987-9219ac4257fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:02.947145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33048eee-d623-464f-8987-9219ac4257fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:02.950594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33048eee-d623-464f-8987-9219ac4257fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:02.952535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33048eee-d623-464f-8987-9219ac4257fd admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:02.975284 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 828/1647] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:02.976605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d52d8c8-d712-4777-acfc-a055cebb211e admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:02.989761 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d52d8c8-d712-4777-acfc-a055cebb211e admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:03.005087 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a6ed1-6b79-4fcc-9c7d-de470fdc1957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:03.006901 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a6ed1-6b79-4fcc-9c7d-de470fdc1957 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:03.007194 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a6ed1-6b79-4fcc-9c7d-de470fdc1957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:03.007476 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a6ed1-6b79-4fcc-9c7d-de470fdc1957 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:03.015141 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a6ed1-6b79-4fcc-9c7d-de470fdc1957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:03.019305 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a6ed1-6b79-4fcc-9c7d-de470fdc1957 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:03.050749 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33048eee-d623-464f-8987-9219ac4257fd admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:03.066545 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 820/1648] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:34:03.069738 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33048eee-d623-464f-8987-9219ac4257fd admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:03.077306 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee28a322-850b-4f9e-bae3-7abfe92adcc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:03.078852 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28a322-850b-4f9e-bae3-7abfe92adcc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:03.079365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28a322-850b-4f9e-bae3-7abfe92adcc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:03.079850 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28a322-850b-4f9e-bae3-7abfe92adcc7 admin admin] PATH_INFO: `/v3/projects/937e87b0003d435281e7611179947b99/users/dcb69d9d9ad44f409116a3c0737eb466/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:03.104816 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee28a322-850b-4f9e-bae3-7abfe92adcc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=937e87b0003d435281e7611179947b99, user_id=dcb69d9d9ad44f409116a3c0737eb466, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:03.110298 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee28a322-850b-4f9e-bae3-7abfe92adcc7 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:03.140176 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 829/1649] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:03 2022] PUT /identity/v3/projects/937e87b0003d435281e7611179947b99/users/dcb69d9d9ad44f409116a3c0737eb466/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:03.142847 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d958450-cb27-4101-9b39-7d5f999c3d3e None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:03.143385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d958450-cb27-4101-9b39-7d5f999c3d3e None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:03.143834 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d958450-cb27-4101-9b39-7d5f999c3d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:03.252644 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-8d958450-cb27-4101-9b39-7d5f999c3d3e None None] MFA Rules not processed for user `dcb69d9d9ad44f409116a3c0737eb466`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:03.254097 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8d958450-cb27-4101-9b39-7d5f999c3d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:03.334495 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:03.396196 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 821/1650] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:03.408179 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee28a322-850b-4f9e-bae3-7abfe92adcc7 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:03.410357 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-ee28a322-850b-4f9e-bae3-7abfe92adcc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:03.504375 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:03.625909 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee28a322-850b-4f9e-bae3-7abfe92adcc7 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:03.634011 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44acf2c-5d66-4bdd-a052-2c9fda195fa1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:03.635017 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44acf2c-5d66-4bdd-a052-2c9fda195fa1 service nova] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:03.638806 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44acf2c-5d66-4bdd-a052-2c9fda195fa1 service nova] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:03.639240 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44acf2c-5d66-4bdd-a052-2c9fda195fa1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:03.654453 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44acf2c-5d66-4bdd-a052-2c9fda195fa1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:03.657244 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44acf2c-5d66-4bdd-a052-2c9fda195fa1 service nova] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:03.682927 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 830/1651] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:04.105163 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d958450-cb27-4101-9b39-7d5f999c3d3e None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:04.320198 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d958450-cb27-4101-9b39-7d5f999c3d3e None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:04.335494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545e3f2-eddc-4a9e-80eb-70980c697ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:04.336136 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545e3f2-eddc-4a9e-80eb-70980c697ce4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:04.336423 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545e3f2-eddc-4a9e-80eb-70980c697ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:04.336699 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545e3f2-eddc-4a9e-80eb-70980c697ce4 admin admin] PATH_INFO: `/v3/users/dcb69d9d9ad44f409116a3c0737eb466` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:04.338512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b545e3f2-eddc-4a9e-80eb-70980c697ce4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcb69d9d9ad44f409116a3c0737eb466)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:04.352515 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b545e3f2-eddc-4a9e-80eb-70980c697ce4 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:04.500740 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b545e3f2-eddc-4a9e-80eb-70980c697ce4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcb69d9d9ad44f409116a3c0737eb466'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:04.526418 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b545e3f2-eddc-4a9e-80eb-70980c697ce4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcb69d9d9ad44f409116a3c0737eb466'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:04.528031 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b545e3f2-eddc-4a9e-80eb-70980c697ce4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcb69d9d9ad44f409116a3c0737eb466'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:04.559491 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b545e3f2-eddc-4a9e-80eb-70980c697ce4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcb69d9d9ad44f409116a3c0737eb466'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:04.560007 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b545e3f2-eddc-4a9e-80eb-70980c697ce4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcb69d9d9ad44f409116a3c0737eb466'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:04.570791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea0f66-16b8-4671-a34c-72075c11dcbf None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:04.571352 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea0f66-16b8-4671-a34c-72075c11dcbf None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:04.571910 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea0f66-16b8-4671-a34c-72075c11dcbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:04.610330 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 822/1652] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:04 2022] DELETE /identity/v3/users/dcb69d9d9ad44f409116a3c0737eb466 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:04.612334 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e6d67-5345-4b97-9a66-ae21413b1776 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:04.612934 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e6d67-5345-4b97-9a66-ae21413b1776 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:04.613931 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e6d67-5345-4b97-9a66-ae21413b1776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:04.638292 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-5cea0f66-16b8-4671-a34c-72075c11dcbf None None] MFA Rules not processed for user `b172f56cbd604342ba6d8169c969e157`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:04.641671 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-5cea0f66-16b8-4671-a34c-72075c11dcbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:04.667447 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-989e6d67-5345-4b97-9a66-ae21413b1776 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:04.670695 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-989e6d67-5345-4b97-9a66-ae21413b1776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:04.818619 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 831/1653] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:04.821323 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:04.829851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ea76e-9766-4bea-b41e-d7b21ea0583d None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:04.830100 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ea76e-9766-4bea-b41e-d7b21ea0583d None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:04.830431 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ea76e-9766-4bea-b41e-d7b21ea0583d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:04.844836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 823/1654] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:04.852742 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-af3ea76e-9766-4bea-b41e-d7b21ea0583d None None] MFA Rules not processed for user `b172f56cbd604342ba6d8169c969e157`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:04.854222 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-af3ea76e-9766-4bea-b41e-d7b21ea0583d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:04.865346 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 832/1655] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:34:04.878464 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e6d67-5345-4b97-9a66-ae21413b1776 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:04.883168 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3ea76e-9766-4bea-b41e-d7b21ea0583d None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:04.884399 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-989e6d67-5345-4b97-9a66-ae21413b1776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:04.893838 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-af3ea76e-9766-4bea-b41e-d7b21ea0583d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:05.023621 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e6d67-5345-4b97-9a66-ae21413b1776 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:05.040039 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caad525-8f76-4bcb-b2d0-143186694462 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:05.042057 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caad525-8f76-4bcb-b2d0-143186694462 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:05.042515 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caad525-8f76-4bcb-b2d0-143186694462 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:05.042896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caad525-8f76-4bcb-b2d0-143186694462 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:05.070482 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caad525-8f76-4bcb-b2d0-143186694462 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:05.075997 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caad525-8f76-4bcb-b2d0-143186694462 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:05.115652 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 824/1656] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:05.124007 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3ea76e-9766-4bea-b41e-d7b21ea0583d None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:05.145736 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c8b24-ec18-4317-b097-e736e4d91c92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:05.146411 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c8b24-ec18-4317-b097-e736e4d91c92 service nova] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:05.146697 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c8b24-ec18-4317-b097-e736e4d91c92 service nova] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:05.146976 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c8b24-ec18-4317-b097-e736e4d91c92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:05.179220 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c8b24-ec18-4317-b097-e736e4d91c92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:05.180811 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c8b24-ec18-4317-b097-e736e4d91c92 service nova] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:05.207572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 833/1657] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:05.781269 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0e1bc-9913-42e2-b9d7-540f916a0ec4 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:05.782258 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0e1bc-9913-42e2-b9d7-540f916a0ec4 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:05.782695 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0e1bc-9913-42e2-b9d7-540f916a0ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:05.866452 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-18c0e1bc-9913-42e2-b9d7-540f916a0ec4 None None] MFA Rules not processed for user `8388303f9098497bb559b3bcb59fab52`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:05.870344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-18c0e1bc-9913-42e2-b9d7-540f916a0ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:06.044461 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 825/1658] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:05 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:06.060349 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c8b24-ec18-4317-b097-e736e4d91c92 service nova] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:06.061871 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-897c8b24-ec18-4317-b097-e736e4d91c92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:06.110261 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c8b24-ec18-4317-b097-e736e4d91c92 service nova] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:06.139047 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a078648-72bc-44dc-a32e-35c781c81462 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:06.139900 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a078648-72bc-44dc-a32e-35c781c81462 service nova] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:06.140443 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a078648-72bc-44dc-a32e-35c781c81462 service nova] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:06.140853 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a078648-72bc-44dc-a32e-35c781c81462 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:06.187292 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a078648-72bc-44dc-a32e-35c781c81462 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:06.190603 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a078648-72bc-44dc-a32e-35c781c81462 service nova] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:06.216195 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 834/1659] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:06 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:06.365835 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187f919-f937-4153-bcf4-10650fd8be89 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:06.366428 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187f919-f937-4153-bcf4-10650fd8be89 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:06.367551 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187f919-f937-4153-bcf4-10650fd8be89 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:06.370002 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 826/1660] 10.222.0.18 () {54 vars in 912 bytes} [Thu Sep 1 16:34:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:34:06.377804 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfafdb0-e5b2-44ef-a61c-fa34f696f667 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:06.378311 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfafdb0-e5b2-44ef-a61c-fa34f696f667 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:06.378771 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfafdb0-e5b2-44ef-a61c-fa34f696f667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:06.449742 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:06.523885 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-6dfafdb0-e5b2-44ef-a61c-fa34f696f667 None None] MFA Rules not processed for user `ffa0f2160a934906b59e6f845edb0776`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:06.525408 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-6dfafdb0-e5b2-44ef-a61c-fa34f696f667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:06.547573 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 835/1661] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Sep 1 16:34:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:34:06.561708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187f919-f937-4153-bcf4-10650fd8be89 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:06.592787 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187f919-f937-4153-bcf4-10650fd8be89 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:06.597426 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a61b1e9-b567-4819-8cc3-fd4c68292781 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:06.598290 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61b1e9-b567-4819-8cc3-fd4c68292781 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:06.598733 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61b1e9-b567-4819-8cc3-fd4c68292781 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:06.599399 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61b1e9-b567-4819-8cc3-fd4c68292781 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:06.619537 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a61b1e9-b567-4819-8cc3-fd4c68292781 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:06.625950 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a61b1e9-b567-4819-8cc3-fd4c68292781 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:06.649604 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 827/1662] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:07.158455 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfafdb0-e5b2-44ef-a61c-fa34f696f667 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:07.159897 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-6dfafdb0-e5b2-44ef-a61c-fa34f696f667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:07.187778 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfafdb0-e5b2-44ef-a61c-fa34f696f667 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:07.202421 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a41321-2bdf-4cc4-bbe6-3b536fa3fc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:07.203380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a41321-2bdf-4cc4-bbe6-3b536fa3fc75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:07.203818 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a41321-2bdf-4cc4-bbe6-3b536fa3fc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:07.204252 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a41321-2bdf-4cc4-bbe6-3b536fa3fc75 admin admin] PATH_INFO: `/v3/users/d5e7228941074b579d0b11334226169c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:07.206112 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a41321-2bdf-4cc4-bbe6-3b536fa3fc75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5e7228941074b579d0b11334226169c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:07.207888 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: WARNING py.warnings [None req-c9a41321-2bdf-4cc4-bbe6-3b536fa3fc75 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:34:07.207888 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: warnings.warn(msg) Sep 01 16:34:07.207888 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: Sep 01 16:34:07.209054 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a41321-2bdf-4cc4-bbe6-3b536fa3fc75 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:07.411872 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c9a41321-2bdf-4cc4-bbe6-3b536fa3fc75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5e7228941074b579d0b11334226169c'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:07.413074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c9a41321-2bdf-4cc4-bbe6-3b536fa3fc75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5e7228941074b579d0b11334226169c'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:07.440175 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c9a41321-2bdf-4cc4-bbe6-3b536fa3fc75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5e7228941074b579d0b11334226169c'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:07.442049 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c9a41321-2bdf-4cc4-bbe6-3b536fa3fc75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5e7228941074b579d0b11334226169c'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:07.503127 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c9a41321-2bdf-4cc4-bbe6-3b536fa3fc75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5e7228941074b579d0b11334226169c'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:07.577928 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 836/1663] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:07 2022] DELETE /identity/v3/users/d5e7228941074b579d0b11334226169c => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:07.582567 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973fbd26-9812-440d-bf9d-c205e1ca609a None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:07.583087 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973fbd26-9812-440d-bf9d-c205e1ca609a None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:07.585040 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973fbd26-9812-440d-bf9d-c205e1ca609a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:07.630183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-973fbd26-9812-440d-bf9d-c205e1ca609a None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:07.640759 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-973fbd26-9812-440d-bf9d-c205e1ca609a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:07.859359 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 828/1664] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:07.897436 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a41321-2bdf-4cc4-bbe6-3b536fa3fc75 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:07.899232 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-c9a41321-2bdf-4cc4-bbe6-3b536fa3fc75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:08.078181 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a41321-2bdf-4cc4-bbe6-3b536fa3fc75 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:08.099118 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae578d-e814-4d2a-b8fc-b3f8f169246a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:08.099962 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae578d-e814-4d2a-b8fc-b3f8f169246a service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:08.100953 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae578d-e814-4d2a-b8fc-b3f8f169246a service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:08.101350 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae578d-e814-4d2a-b8fc-b3f8f169246a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:08.129156 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae578d-e814-4d2a-b8fc-b3f8f169246a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:08.131051 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae578d-e814-4d2a-b8fc-b3f8f169246a service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:08.146910 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 837/1665] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:08.196611 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973fbd26-9812-440d-bf9d-c205e1ca609a None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:08.198807 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-973fbd26-9812-440d-bf9d-c205e1ca609a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:08.238365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973fbd26-9812-440d-bf9d-c205e1ca609a None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:08.270676 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd927889-cbb9-4ae4-8914-9ebd7243e51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:08.273626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd927889-cbb9-4ae4-8914-9ebd7243e51c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:08.273912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd927889-cbb9-4ae4-8914-9ebd7243e51c admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:08.274191 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd927889-cbb9-4ae4-8914-9ebd7243e51c admin admin] PATH_INFO: `/v3/users/546082aeb45e4ebb9b4619feb41b50fd` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:08.279067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd927889-cbb9-4ae4-8914-9ebd7243e51c admin admin] RBAC: Authorizing `identity:delete_user(user_id=546082aeb45e4ebb9b4619feb41b50fd)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:08.286402 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd927889-cbb9-4ae4-8914-9ebd7243e51c admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:08.321433 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae578d-e814-4d2a-b8fc-b3f8f169246a service neutron] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:08.325489 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-85ae578d-e814-4d2a-b8fc-b3f8f169246a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:08.372342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae578d-e814-4d2a-b8fc-b3f8f169246a service neutron] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:08.385691 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373725b-9f27-4e01-8371-25111c734647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:08.390847 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373725b-9f27-4e01-8371-25111c734647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:08.391438 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373725b-9f27-4e01-8371-25111c734647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:08.394209 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373725b-9f27-4e01-8371-25111c734647 admin admin] PATH_INFO: `/v3/projects/937e87b0003d435281e7611179947b99` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:08.399567 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6373725b-9f27-4e01-8371-25111c734647 admin admin] RBAC: Authorizing `identity:delete_project(project_id=937e87b0003d435281e7611179947b99)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:08.403595 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:08.409992 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6373725b-9f27-4e01-8371-25111c734647 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:08.511047 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-cd927889-cbb9-4ae4-8914-9ebd7243e51c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '546082aeb45e4ebb9b4619feb41b50fd'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:08.546900 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-cd927889-cbb9-4ae4-8914-9ebd7243e51c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '546082aeb45e4ebb9b4619feb41b50fd'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:08.547882 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-cd927889-cbb9-4ae4-8914-9ebd7243e51c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '546082aeb45e4ebb9b4619feb41b50fd'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:08.569888 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-6373725b-9f27-4e01-8371-25111c734647 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '937e87b0003d435281e7611179947b99', 'request_id': 'req-6373725b-9f27-4e01-8371-25111c734647'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:08.573213 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-cd927889-cbb9-4ae4-8914-9ebd7243e51c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '546082aeb45e4ebb9b4619feb41b50fd'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:08.574355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-cd927889-cbb9-4ae4-8914-9ebd7243e51c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '546082aeb45e4ebb9b4619feb41b50fd'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:08.584934 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-6373725b-9f27-4e01-8371-25111c734647 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '937e87b0003d435281e7611179947b99', 'request_id': 'req-6373725b-9f27-4e01-8371-25111c734647'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:08.600280 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-6373725b-9f27-4e01-8371-25111c734647 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '937e87b0003d435281e7611179947b99', 'request_id': 'req-6373725b-9f27-4e01-8371-25111c734647'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:08.602613 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:08.612987 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 829/1666] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:08 2022] DELETE /identity/v3/users/546082aeb45e4ebb9b4619feb41b50fd => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:08.615136 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd927889-cbb9-4ae4-8914-9ebd7243e51c admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:08.619252 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-cd927889-cbb9-4ae4-8914-9ebd7243e51c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:08.629330 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-6373725b-9f27-4e01-8371-25111c734647 admin admin] The token cache is being invalidate because project 937e87b0003d435281e7611179947b99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:08.632526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-6373725b-9f27-4e01-8371-25111c734647 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:08.633736 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-6373725b-9f27-4e01-8371-25111c734647 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:08.637411 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 838/1667] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:08 2022] DELETE /identity/v3/projects/937e87b0003d435281e7611179947b99 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:08.639728 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373725b-9f27-4e01-8371-25111c734647 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:08.644904 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-6373725b-9f27-4e01-8371-25111c734647 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:08.803374 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd927889-cbb9-4ae4-8914-9ebd7243e51c admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:08.810887 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373725b-9f27-4e01-8371-25111c734647 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:08.815744 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d337c8-9d03-4ebc-8df5-9de340ccc873 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ddfb151bbf844efae73ec1990e7eaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:08.816207 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d337c8-9d03-4ebc-8df5-9de340ccc873 service placement] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:08.817497 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d337c8-9d03-4ebc-8df5-9de340ccc873 service placement] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:08.817801 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d337c8-9d03-4ebc-8df5-9de340ccc873 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:08.823025 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-82d337c8-9d03-4ebc-8df5-9de340ccc873 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:08.847212 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d337c8-9d03-4ebc-8df5-9de340ccc873 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:08.852171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d337c8-9d03-4ebc-8df5-9de340ccc873 service placement] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:08.889806 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 830/1668] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:08.911000 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e699a-212e-4b6c-aea3-1a0f73376cfd None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:08.911327 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e699a-212e-4b6c-aea3-1a0f73376cfd None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:08.912769 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e699a-212e-4b6c-aea3-1a0f73376cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:08.991423 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-ea4e699a-212e-4b6c-aea3-1a0f73376cfd None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:08.998995 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ea4e699a-212e-4b6c-aea3-1a0f73376cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:09.018926 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a83e8-de22-4572-b6e6-66e0588fcc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:09.020470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a83e8-de22-4572-b6e6-66e0588fcc1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:09.022138 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a83e8-de22-4572-b6e6-66e0588fcc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:09.023043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a83e8-de22-4572-b6e6-66e0588fcc1e admin admin] PATH_INFO: `/v3/users/d93e5e6e52fd4b98b36aee5b18cfb249` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:09.026654 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a83e8-de22-4572-b6e6-66e0588fcc1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d93e5e6e52fd4b98b36aee5b18cfb249)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:09.036558 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a83e8-de22-4572-b6e6-66e0588fcc1e admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:09.198862 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-343a83e8-de22-4572-b6e6-66e0588fcc1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd93e5e6e52fd4b98b36aee5b18cfb249'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:09.200118 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-343a83e8-de22-4572-b6e6-66e0588fcc1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd93e5e6e52fd4b98b36aee5b18cfb249'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:09.216607 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 831/1669] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:09.223253 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-343a83e8-de22-4572-b6e6-66e0588fcc1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd93e5e6e52fd4b98b36aee5b18cfb249'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:09.223961 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-343a83e8-de22-4572-b6e6-66e0588fcc1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd93e5e6e52fd4b98b36aee5b18cfb249'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:09.234960 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2ef72-95d9-4a0b-9066-8d29ced575e2 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:09.235262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2ef72-95d9-4a0b-9066-8d29ced575e2 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:09.235533 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2ef72-95d9-4a0b-9066-8d29ced575e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:09.251920 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-343a83e8-de22-4572-b6e6-66e0588fcc1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd93e5e6e52fd4b98b36aee5b18cfb249'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:09.268477 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 839/1670] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:08 2022] DELETE /identity/v3/users/d93e5e6e52fd4b98b36aee5b18cfb249 => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:09.269742 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a83e8-de22-4572-b6e6-66e0588fcc1e admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:09.274068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-343a83e8-de22-4572-b6e6-66e0588fcc1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:09.302924 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:09.309303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-b9a2ef72-95d9-4a0b-9066-8d29ced575e2 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:09.314966 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b9a2ef72-95d9-4a0b-9066-8d29ced575e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:09.359414 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:09.427548 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a83e8-de22-4572-b6e6-66e0588fcc1e admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:09.439497 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576f76a-7a53-4862-8efb-609f1d57eec8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:09.440147 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576f76a-7a53-4862-8efb-609f1d57eec8 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:09.442645 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576f76a-7a53-4862-8efb-609f1d57eec8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:09.443207 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576f76a-7a53-4862-8efb-609f1d57eec8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:09.534368 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 832/1671] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:09.538500 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c375a7-4a16-4021-b0f4-e615d1edee7f None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:09.538833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c375a7-4a16-4021-b0f4-e615d1edee7f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:09.539124 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c375a7-4a16-4021-b0f4-e615d1edee7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:09.581505 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-22c375a7-4a16-4021-b0f4-e615d1edee7f None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:09.582809 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-22c375a7-4a16-4021-b0f4-e615d1edee7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:09.618566 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 833/1672] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:09.619863 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7622151b-78df-478c-8d30-3b5acd879248 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:09.620151 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7622151b-78df-478c-8d30-3b5acd879248 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:09.620427 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7622151b-78df-478c-8d30-3b5acd879248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:09.660576 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-7622151b-78df-478c-8d30-3b5acd879248 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:09.662080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7622151b-78df-478c-8d30-3b5acd879248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:09.686881 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 834/1673] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:09.686881 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7622151b-78df-478c-8d30-3b5acd879248 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:09.700467 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7622151b-78df-478c-8d30-3b5acd879248 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:09.716124 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05819183-2cb8-42f7-9b62-1730ae1d4b71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:09.717036 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05819183-2cb8-42f7-9b62-1730ae1d4b71 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:09.722532 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05819183-2cb8-42f7-9b62-1730ae1d4b71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:09.722532 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05819183-2cb8-42f7-9b62-1730ae1d4b71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:09.738683 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a576f76a-7a53-4862-8efb-609f1d57eec8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:09.744308 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a576f76a-7a53-4862-8efb-609f1d57eec8 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:09.747895 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05819183-2cb8-42f7-9b62-1730ae1d4b71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:09.753424 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05819183-2cb8-42f7-9b62-1730ae1d4b71 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:09.779716 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 835/1674] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:09.780419 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05819183-2cb8-42f7-9b62-1730ae1d4b71 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:09.791381 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 840/1675] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:09.797636 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e80c3-f8be-4563-ab55-113b33f506a3 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:09.797929 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e80c3-f8be-4563-ab55-113b33f506a3 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:09.798215 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e80c3-f8be-4563-ab55-113b33f506a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:09.806832 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05819183-2cb8-42f7-9b62-1730ae1d4b71 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:09.813727 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f395e-c3f1-414c-8bdc-7edf9fbca8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:09.814378 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f395e-c3f1-414c-8bdc-7edf9fbca8ab admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:09.814656 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f395e-c3f1-414c-8bdc-7edf9fbca8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:09.814930 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f395e-c3f1-414c-8bdc-7edf9fbca8ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:09.815582 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059f395e-c3f1-414c-8bdc-7edf9fbca8ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:34:09.815891 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059f395e-c3f1-414c-8bdc-7edf9fbca8ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:09.817492 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059f395e-c3f1-414c-8bdc-7edf9fbca8ab admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:09.848150 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 836/1676] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:34:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:34:09.853069 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f395e-c3f1-414c-8bdc-7edf9fbca8ab admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:09.873430 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f395e-c3f1-414c-8bdc-7edf9fbca8ab admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:09.882153 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0fdc3f-abcb-4bd8-a37a-f11638ca9ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:09.882562 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fdc3f-abcb-4bd8-a37a-f11638ca9ee7 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:09.882921 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fdc3f-abcb-4bd8-a37a-f11638ca9ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:09.883214 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fdc3f-abcb-4bd8-a37a-f11638ca9ee7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:09.883735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0fdc3f-abcb-4bd8-a37a-f11638ca9ee7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:09.891483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0fdc3f-abcb-4bd8-a37a-f11638ca9ee7 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:09.921291 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-1a7e80c3-f8be-4563-ab55-113b33f506a3 None None] MFA Rules not processed for user `73624e79ccdd4782beddf663bb6d02e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:09.922577 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-1a7e80c3-f8be-4563-ab55-113b33f506a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:09.961862 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 837/1677] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:09 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:34:09.965028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0fdc3f-abcb-4bd8-a37a-f11638ca9ee7 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:10.132587 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 841/1678] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:10.148399 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e80c3-f8be-4563-ab55-113b33f506a3 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:10.150062 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-1a7e80c3-f8be-4563-ab55-113b33f506a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:10.191794 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0fdc3f-abcb-4bd8-a37a-f11638ca9ee7 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:10.215452 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ada9bb-7401-4339-922d-d66c91b780a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:10.216099 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ada9bb-7401-4339-922d-d66c91b780a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:10.216428 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ada9bb-7401-4339-922d-d66c91b780a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:10.216639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ada9bb-7401-4339-922d-d66c91b780a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:10.217317 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ada9bb-7401-4339-922d-d66c91b780a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:10.219272 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ada9bb-7401-4339-922d-d66c91b780a0 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:10.274809 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 838/1679] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:09 2022] POST /identity/v3/users => generated 288 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:10.279751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ada9bb-7401-4339-922d-d66c91b780a0 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:10.310271 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:10.318781 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ada9bb-7401-4339-922d-d66c91b780a0 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:10.338972 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e675f69-f535-431f-b3f0-13d923228b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:10.339655 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e675f69-f535-431f-b3f0-13d923228b97 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:10.339942 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e675f69-f535-431f-b3f0-13d923228b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:10.340217 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e675f69-f535-431f-b3f0-13d923228b97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:10.341677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e675f69-f535-431f-b3f0-13d923228b97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:10.343308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e675f69-f535-431f-b3f0-13d923228b97 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:10.391625 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 839/1680] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:34:10.391782 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e80c3-f8be-4563-ab55-113b33f506a3 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:10.392979 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e675f69-f535-431f-b3f0-13d923228b97 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:10.419686 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e675f69-f535-431f-b3f0-13d923228b97 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:10.432036 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e7bfb-12cd-4fcd-901a-f232f5732e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:10.432917 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e7bfb-12cd-4fcd-901a-f232f5732e30 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:10.435449 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e7bfb-12cd-4fcd-901a-f232f5732e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:10.436477 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e7bfb-12cd-4fcd-901a-f232f5732e30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:10.436706 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2e7bfb-12cd-4fcd-901a-f232f5732e30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:10.440120 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2e7bfb-12cd-4fcd-901a-f232f5732e30 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:10.461975 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 840/1681] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:34:10.475137 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e7bfb-12cd-4fcd-901a-f232f5732e30 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:10.524751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e7bfb-12cd-4fcd-901a-f232f5732e30 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:10.545158 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6936853-6e8c-40e1-bd8d-4f0cdcf9de74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:10.546126 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6936853-6e8c-40e1-bd8d-4f0cdcf9de74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:10.546456 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6936853-6e8c-40e1-bd8d-4f0cdcf9de74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:10.547771 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6936853-6e8c-40e1-bd8d-4f0cdcf9de74 admin admin] PATH_INFO: `/v3/projects/0a1b9052f76f45fdb904e92d4bd3949f/users/af6d0bcb31a649c790421aa7248ca30e/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:10.575798 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6936853-6e8c-40e1-bd8d-4f0cdcf9de74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a1b9052f76f45fdb904e92d4bd3949f, user_id=af6d0bcb31a649c790421aa7248ca30e, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:10.581647 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6936853-6e8c-40e1-bd8d-4f0cdcf9de74 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:10.608256 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead5774-5cc6-4f18-a827-108e9acba61c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:10.608914 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead5774-5cc6-4f18-a827-108e9acba61c service nova] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:10.614579 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead5774-5cc6-4f18-a827-108e9acba61c service nova] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:10.614968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead5774-5cc6-4f18-a827-108e9acba61c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:10.625090 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 841/1682] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:10 2022] PUT /identity/v3/projects/0a1b9052f76f45fdb904e92d4bd3949f/users/af6d0bcb31a649c790421aa7248ca30e/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:10.636027 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7135c23d-8803-4fea-9300-05ff864db82d None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:10.637272 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7135c23d-8803-4fea-9300-05ff864db82d None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:10.637556 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7135c23d-8803-4fea-9300-05ff864db82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:10.671717 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:10.757944 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-7135c23d-8803-4fea-9300-05ff864db82d None None] MFA Rules not processed for user `af6d0bcb31a649c790421aa7248ca30e`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:10.760646 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7135c23d-8803-4fea-9300-05ff864db82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:10.863523 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead5774-5cc6-4f18-a827-108e9acba61c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:10.865794 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead5774-5cc6-4f18-a827-108e9acba61c service nova] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:10.882647 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 842/1683] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 735 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:34:10.884035 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead5774-5cc6-4f18-a827-108e9acba61c service nova] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:10.885816 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-6ead5774-5cc6-4f18-a827-108e9acba61c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:10.936932 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead5774-5cc6-4f18-a827-108e9acba61c service nova] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:11.014676 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 842/1684] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:11.019331 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7135c23d-8803-4fea-9300-05ff864db82d None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:11.139789 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d1e1b-7327-4835-bd6a-f0ff58daee5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:11.140810 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d1e1b-7327-4835-bd6a-f0ff58daee5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:11.141158 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d1e1b-7327-4835-bd6a-f0ff58daee5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:11.143368 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d1e1b-7327-4835-bd6a-f0ff58daee5e admin admin] PATH_INFO: `/v3/projects/170a066ef9584da0915a40ff4379c985` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:11.144973 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426d1e1b-7327-4835-bd6a-f0ff58daee5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=170a066ef9584da0915a40ff4379c985)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:11.151052 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426d1e1b-7327-4835-bd6a-f0ff58daee5e admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:11.198633 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7135c23d-8803-4fea-9300-05ff864db82d None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:11.210317 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dab025-0f94-48c4-840a-9090acc6e555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:11.210761 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dab025-0f94-48c4-840a-9090acc6e555 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:11.211046 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dab025-0f94-48c4-840a-9090acc6e555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:11.213012 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dab025-0f94-48c4-840a-9090acc6e555 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:11.215378 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dab025-0f94-48c4-840a-9090acc6e555 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:11.218322 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dab025-0f94-48c4-840a-9090acc6e555 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:11.244106 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:11.272406 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-426d1e1b-7327-4835-bd6a-f0ff58daee5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '170a066ef9584da0915a40ff4379c985', 'request_id': 'req-426d1e1b-7327-4835-bd6a-f0ff58daee5e'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:11.306121 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 843/1685] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:11 2022] POST /identity/v3/projects => generated 377 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:11.311024 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dab025-0f94-48c4-840a-9090acc6e555 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:11.324124 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-426d1e1b-7327-4835-bd6a-f0ff58daee5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '170a066ef9584da0915a40ff4379c985', 'request_id': 'req-426d1e1b-7327-4835-bd6a-f0ff58daee5e'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:11.335247 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-426d1e1b-7327-4835-bd6a-f0ff58daee5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '170a066ef9584da0915a40ff4379c985', 'request_id': 'req-426d1e1b-7327-4835-bd6a-f0ff58daee5e'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:11.369255 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-426d1e1b-7327-4835-bd6a-f0ff58daee5e admin admin] The token cache is being invalidate because project 170a066ef9584da0915a40ff4379c985 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:11.369579 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-426d1e1b-7327-4835-bd6a-f0ff58daee5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:11.371025 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-426d1e1b-7327-4835-bd6a-f0ff58daee5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:11.379142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 843/1686] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:10 2022] DELETE /identity/v3/projects/170a066ef9584da0915a40ff4379c985 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:11.401563 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32981a16-b183-4de5-98f6-a800242d0a6d None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:11.401898 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32981a16-b183-4de5-98f6-a800242d0a6d None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:11.402791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32981a16-b183-4de5-98f6-a800242d0a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:11.490732 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-32981a16-b183-4de5-98f6-a800242d0a6d None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:11.498112 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-32981a16-b183-4de5-98f6-a800242d0a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:11.504238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dab025-0f94-48c4-840a-9090acc6e555 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:11.517936 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7f2f4-f5a3-4525-bd6e-1b3c6f5cd6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:11.519515 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7f2f4-f5a3-4525-bd6e-1b3c6f5cd6f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:11.519515 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7f2f4-f5a3-4525-bd6e-1b3c6f5cd6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:11.519515 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7f2f4-f5a3-4525-bd6e-1b3c6f5cd6f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:11.520054 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded7f2f4-f5a3-4525-bd6e-1b3c6f5cd6f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:11.524018 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded7f2f4-f5a3-4525-bd6e-1b3c6f5cd6f9 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:11.547057 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 844/1687] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:11.550685 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ed4b3-87d8-40db-91e3-a8b799a921f3 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:11.550980 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ed4b3-87d8-40db-91e3-a8b799a921f3 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:11.551269 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ed4b3-87d8-40db-91e3-a8b799a921f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:11.585179 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 844/1688] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:11 2022] POST /identity/v3/users => generated 295 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:11.586134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7f2f4-f5a3-4525-bd6e-1b3c6f5cd6f9 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:11.592914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ded7f2f4-f5a3-4525-bd6e-1b3c6f5cd6f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:11.600552 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-2d4ed4b3-87d8-40db-91e3-a8b799a921f3 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:11.602151 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-2d4ed4b3-87d8-40db-91e3-a8b799a921f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:11.616413 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7f2f4-f5a3-4525-bd6e-1b3c6f5cd6f9 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:11.625381 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 845/1689] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:11.629262 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ed4b3-87d8-40db-91e3-a8b799a921f3 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:11.633606 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbfa252-9bc6-435f-86e1-0208a45f9941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:11.634531 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbfa252-9bc6-435f-86e1-0208a45f9941 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:11.634816 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbfa252-9bc6-435f-86e1-0208a45f9941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:11.634816 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbfa252-9bc6-435f-86e1-0208a45f9941 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:11.635471 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbfa252-9bc6-435f-86e1-0208a45f9941 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:11.637132 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbfa252-9bc6-435f-86e1-0208a45f9941 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:11.638908 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:11.652672 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ed4b3-87d8-40db-91e3-a8b799a921f3 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:11.658459 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0bec39-ae1e-4059-8f3f-a45abb1d09a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:11.659105 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bec39-ae1e-4059-8f3f-a45abb1d09a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:11.659378 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bec39-ae1e-4059-8f3f-a45abb1d09a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:11.659657 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bec39-ae1e-4059-8f3f-a45abb1d09a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:11.660326 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0bec39-ae1e-4059-8f3f-a45abb1d09a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:34:11.660615 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0bec39-ae1e-4059-8f3f-a45abb1d09a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:11.665021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0bec39-ae1e-4059-8f3f-a45abb1d09a3 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:11.666976 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 845/1690] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:11.668054 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbfa252-9bc6-435f-86e1-0208a45f9941 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:11.701320 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 846/1691] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:34:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:34:11.701539 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbfa252-9bc6-435f-86e1-0208a45f9941 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:11.704036 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0bec39-ae1e-4059-8f3f-a45abb1d09a3 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:11.719127 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0bec39-ae1e-4059-8f3f-a45abb1d09a3 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:11.729769 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d723eaa-1dfa-4cad-8d13-67c76bef9b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:11.729769 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d723eaa-1dfa-4cad-8d13-67c76bef9b1b admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:11.730121 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d723eaa-1dfa-4cad-8d13-67c76bef9b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:11.730653 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d723eaa-1dfa-4cad-8d13-67c76bef9b1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:11.731302 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d723eaa-1dfa-4cad-8d13-67c76bef9b1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:11.731863 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e762402-803b-4357-a548-a8e9060ea756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:11.734483 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d723eaa-1dfa-4cad-8d13-67c76bef9b1b admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:11.737129 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e762402-803b-4357-a548-a8e9060ea756 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:11.737485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e762402-803b-4357-a548-a8e9060ea756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:11.738508 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e762402-803b-4357-a548-a8e9060ea756 admin admin] PATH_INFO: `/v3/projects/710f523c46724172a8cc4726770c525a/users/7b5bb04abbbf4d0781ca3a47cd882555/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:11.788844 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e762402-803b-4357-a548-a8e9060ea756 admin admin] RBAC: Authorizing `identity:create_grant(project_id=710f523c46724172a8cc4726770c525a, user_id=7b5bb04abbbf4d0781ca3a47cd882555, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:11.796932 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e762402-803b-4357-a548-a8e9060ea756 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:11.821527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 847/1692] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:11 2022] POST /identity/v3/projects => generated 371 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:34:11.827192 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d723eaa-1dfa-4cad-8d13-67c76bef9b1b admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:11.836372 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 846/1693] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:11 2022] PUT /identity/v3/projects/710f523c46724172a8cc4726770c525a/users/7b5bb04abbbf4d0781ca3a47cd882555/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:11.837932 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e762402-803b-4357-a548-a8e9060ea756 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:11.925020 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e762402-803b-4357-a548-a8e9060ea756 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:11.958789 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d723eaa-1dfa-4cad-8d13-67c76bef9b1b admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:11.968545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff61d5-94ff-4493-92ef-398e2c1e150f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:11.969187 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff61d5-94ff-4493-92ef-398e2c1e150f admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:11.969474 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff61d5-94ff-4493-92ef-398e2c1e150f admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:11.969745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff61d5-94ff-4493-92ef-398e2c1e150f admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:11.970468 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff61d5-94ff-4493-92ef-398e2c1e150f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:11.972498 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff61d5-94ff-4493-92ef-398e2c1e150f admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:12.000722 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 848/1694] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:11 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:12.004642 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff61d5-94ff-4493-92ef-398e2c1e150f admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:12.040445 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff61d5-94ff-4493-92ef-398e2c1e150f admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:12.061626 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b82dd7-250e-458c-95bc-e167d6af9476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:12.062990 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b82dd7-250e-458c-95bc-e167d6af9476 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:12.063298 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b82dd7-250e-458c-95bc-e167d6af9476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:12.063579 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b82dd7-250e-458c-95bc-e167d6af9476 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:12.064248 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b82dd7-250e-458c-95bc-e167d6af9476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:12.065975 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b82dd7-250e-458c-95bc-e167d6af9476 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:12.100609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 849/1695] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:34:12.104404 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b82dd7-250e-458c-95bc-e167d6af9476 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:12.122010 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b82dd7-250e-458c-95bc-e167d6af9476 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:12.127832 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a731f-97ec-452c-8d38-8d4cfa6b831b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:12.128825 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a731f-97ec-452c-8d38-8d4cfa6b831b admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:12.129274 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a731f-97ec-452c-8d38-8d4cfa6b831b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:12.129644 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a731f-97ec-452c-8d38-8d4cfa6b831b admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:12.131077 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7a731f-97ec-452c-8d38-8d4cfa6b831b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:12.138287 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ffccd-7ca9-4952-87bc-af26dc37ce1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:12.139013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ffccd-7ca9-4952-87bc-af26dc37ce1d admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:12.139382 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ffccd-7ca9-4952-87bc-af26dc37ce1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:12.139756 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ffccd-7ca9-4952-87bc-af26dc37ce1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:12.140815 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ffccd-7ca9-4952-87bc-af26dc37ce1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:12.142359 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7a731f-97ec-452c-8d38-8d4cfa6b831b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:12.146337 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ffccd-7ca9-4952-87bc-af26dc37ce1d admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:12.178344 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 850/1696] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:12.183296 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ffccd-7ca9-4952-87bc-af26dc37ce1d admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:12.191530 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 847/1697] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:12.198583 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a731f-97ec-452c-8d38-8d4cfa6b831b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:12.229839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a731f-97ec-452c-8d38-8d4cfa6b831b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:12.231490 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ffccd-7ca9-4952-87bc-af26dc37ce1d admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:12.244360 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549e774-84ef-4929-b5d0-9d86d3b0a819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:12.246296 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549e774-84ef-4929-b5d0-9d86d3b0a819 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:12.247512 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549e774-84ef-4929-b5d0-9d86d3b0a819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:12.248169 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17b954-c5ae-4639-be3d-40f8b4dde903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:12.248857 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549e774-84ef-4929-b5d0-9d86d3b0a819 admin admin] PATH_INFO: `/v3/projects/a7bf65301a364090aa54e0eea3f332c6/users/7ab0658d5d334fbe8100d3b340216ec2/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:12.250140 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17b954-c5ae-4639-be3d-40f8b4dde903 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:12.250475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17b954-c5ae-4639-be3d-40f8b4dde903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:12.250752 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17b954-c5ae-4639-be3d-40f8b4dde903 admin admin] PATH_INFO: `/v3/projects/710f523c46724172a8cc4726770c525a/users/7b5bb04abbbf4d0781ca3a47cd882555/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:12.257490 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17b954-c5ae-4639-be3d-40f8b4dde903 admin admin] RBAC: Authorizing `identity:create_grant(project_id=710f523c46724172a8cc4726770c525a, user_id=7b5bb04abbbf4d0781ca3a47cd882555, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:12.261000 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17b954-c5ae-4639-be3d-40f8b4dde903 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:12.275759 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 848/1698] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:12 2022] PUT /identity/v3/projects/710f523c46724172a8cc4726770c525a/users/7b5bb04abbbf4d0781ca3a47cd882555/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:12.285990 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc6085-bf19-4ad6-b018-13f32abcb481 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:12.286336 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc6085-bf19-4ad6-b018-13f32abcb481 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:12.286579 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc6085-bf19-4ad6-b018-13f32abcb481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:12.302295 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549e774-84ef-4929-b5d0-9d86d3b0a819 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7bf65301a364090aa54e0eea3f332c6, user_id=7ab0658d5d334fbe8100d3b340216ec2, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:12.315581 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549e774-84ef-4929-b5d0-9d86d3b0a819 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:12.348106 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 851/1699] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:12 2022] PUT /identity/v3/projects/a7bf65301a364090aa54e0eea3f332c6/users/7ab0658d5d334fbe8100d3b340216ec2/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:12.360077 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d148d-2da4-4b85-913e-f67fe106c393 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:12.360363 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d148d-2da4-4b85-913e-f67fe106c393 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:12.360815 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d148d-2da4-4b85-913e-f67fe106c393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:12.372581 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-27bc6085-bf19-4ad6-b018-13f32abcb481 None None] MFA Rules not processed for user `7b5bb04abbbf4d0781ca3a47cd882555`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:12.373834 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-27bc6085-bf19-4ad6-b018-13f32abcb481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:12.449194 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-680d148d-2da4-4b85-913e-f67fe106c393 None None] MFA Rules not processed for user `7ab0658d5d334fbe8100d3b340216ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:12.450784 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-680d148d-2da4-4b85-913e-f67fe106c393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:12.460164 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=89,keystone:DELETE=5,keystone:INSERT=6 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:12.481003 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:12.513552 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 849/1700] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:12 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:12.523519 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc6085-bf19-4ad6-b018-13f32abcb481 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:12.525032 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-27bc6085-bf19-4ad6-b018-13f32abcb481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:12.610142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 852/1701] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:34:12.615279 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d148d-2da4-4b85-913e-f67fe106c393 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:12.702000 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d148d-2da4-4b85-913e-f67fe106c393 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:12.709407 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1016c74c-c241-4016-a135-f3dfc834e1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:12.710289 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016c74c-c241-4016-a135-f3dfc834e1f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:12.710673 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016c74c-c241-4016-a135-f3dfc834e1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:12.711162 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016c74c-c241-4016-a135-f3dfc834e1f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:12.712045 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1016c74c-c241-4016-a135-f3dfc834e1f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:12.714531 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1016c74c-c241-4016-a135-f3dfc834e1f8 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:12.720914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc6085-bf19-4ad6-b018-13f32abcb481 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:12.732285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c56965-0f2f-416f-b845-838afa1b6e9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:12.732924 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c56965-0f2f-416f-b845-838afa1b6e9e service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:12.733204 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c56965-0f2f-416f-b845-838afa1b6e9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:12.733475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c56965-0f2f-416f-b845-838afa1b6e9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:12.735857 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-80c56965-0f2f-416f-b845-838afa1b6e9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:12.759053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 853/1702] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:12 2022] POST /identity/v3/projects => generated 367 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:34:12.760069 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1016c74c-c241-4016-a135-f3dfc834e1f8 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:12.766866 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-1016c74c-c241-4016-a135-f3dfc834e1f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:12.884580 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c56965-0f2f-416f-b845-838afa1b6e9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:12.887739 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c56965-0f2f-416f-b845-838afa1b6e9e service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:12.920472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 850/1703] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:12.921323 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c56965-0f2f-416f-b845-838afa1b6e9e service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:12.949382 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c56965-0f2f-416f-b845-838afa1b6e9e service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:13.004547 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1016c74c-c241-4016-a135-f3dfc834e1f8 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:13.023189 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee57a6-5b55-422f-b4b4-850944a0cb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:13.024079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee57a6-5b55-422f-b4b4-850944a0cb2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:13.024321 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee57a6-5b55-422f-b4b4-850944a0cb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:13.024681 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee57a6-5b55-422f-b4b4-850944a0cb2f admin admin] PATH_INFO: `/v3/projects/07369ffa37c54449978dfb5ef24150ed` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:13.025937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee57a6-5b55-422f-b4b4-850944a0cb2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=07369ffa37c54449978dfb5ef24150ed)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:13.031428 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee57a6-5b55-422f-b4b4-850944a0cb2f admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:13.115913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb72382b-9aeb-4538-9d28-2e195973b94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:13.116541 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72382b-9aeb-4538-9d28-2e195973b94b admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:13.116821 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72382b-9aeb-4538-9d28-2e195973b94b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:13.117098 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72382b-9aeb-4538-9d28-2e195973b94b admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:13.117727 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb72382b-9aeb-4538-9d28-2e195973b94b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:13.119665 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb72382b-9aeb-4538-9d28-2e195973b94b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:13.125755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:13.163663 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-e9ee57a6-5b55-422f-b4b4-850944a0cb2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07369ffa37c54449978dfb5ef24150ed', 'request_id': 'req-e9ee57a6-5b55-422f-b4b4-850944a0cb2f'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:13.173557 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 851/1704] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:12 2022] POST /identity/v3/users => generated 290 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:13.174549 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb72382b-9aeb-4538-9d28-2e195973b94b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:13.176922 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-e9ee57a6-5b55-422f-b4b4-850944a0cb2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07369ffa37c54449978dfb5ef24150ed', 'request_id': 'req-e9ee57a6-5b55-422f-b4b4-850944a0cb2f'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:13.190514 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-e9ee57a6-5b55-422f-b4b4-850944a0cb2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07369ffa37c54449978dfb5ef24150ed', 'request_id': 'req-e9ee57a6-5b55-422f-b4b4-850944a0cb2f'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:13.193585 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-bb72382b-9aeb-4538-9d28-2e195973b94b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:13.228644 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb72382b-9aeb-4538-9d28-2e195973b94b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:13.232397 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-e9ee57a6-5b55-422f-b4b4-850944a0cb2f admin admin] The token cache is being invalidate because project 07369ffa37c54449978dfb5ef24150ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:13.232779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-e9ee57a6-5b55-422f-b4b4-850944a0cb2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:13.238760 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-e9ee57a6-5b55-422f-b4b4-850944a0cb2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:13.240292 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 854/1705] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:12 2022] DELETE /identity/v3/projects/07369ffa37c54449978dfb5ef24150ed => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:13.241470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6b95c-0de5-4445-a8d0-0452cdf7b9e1 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:13.241754 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6b95c-0de5-4445-a8d0-0452cdf7b9e1 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:13.242028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6b95c-0de5-4445-a8d0-0452cdf7b9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:13.327994 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:13.369536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-6eb6b95c-0de5-4445-a8d0-0452cdf7b9e1 None None] MFA Rules not processed for user `a5b3e53fd1314e17ad28c42a44e8eff3`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:13.372506 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-6eb6b95c-0de5-4445-a8d0-0452cdf7b9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:13.407637 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617167af-49c5-4210-8780-cce13dd26753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:13.408302 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617167af-49c5-4210-8780-cce13dd26753 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:13.409321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617167af-49c5-4210-8780-cce13dd26753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:13.409321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617167af-49c5-4210-8780-cce13dd26753 admin admin] PATH_INFO: `/v3/projects/ed187feab3f94be38411d79b7600d191` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:13.412694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617167af-49c5-4210-8780-cce13dd26753 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed187feab3f94be38411d79b7600d191)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:13.413733 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: WARNING py.warnings [None req-617167af-49c5-4210-8780-cce13dd26753 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:34:13.413733 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: warnings.warn(msg) Sep 01 16:34:13.413733 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: Sep 01 16:34:13.414980 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617167af-49c5-4210-8780-cce13dd26753 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:13.502500 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 855/1706] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Sep 1 16:34:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:13.505339 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972fcf9-8f1d-4cfa-b962-3447ae904a4f None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:13.506108 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972fcf9-8f1d-4cfa-b962-3447ae904a4f None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:13.506431 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972fcf9-8f1d-4cfa-b962-3447ae904a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:13.533452 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-617167af-49c5-4210-8780-cce13dd26753 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed187feab3f94be38411d79b7600d191', 'request_id': 'req-617167af-49c5-4210-8780-cce13dd26753'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:13.569207 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-617167af-49c5-4210-8780-cce13dd26753 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed187feab3f94be38411d79b7600d191', 'request_id': 'req-617167af-49c5-4210-8780-cce13dd26753'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:13.577866 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-617167af-49c5-4210-8780-cce13dd26753 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed187feab3f94be38411d79b7600d191', 'request_id': 'req-617167af-49c5-4210-8780-cce13dd26753'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:13.588490 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-d972fcf9-8f1d-4cfa-b962-3447ae904a4f None None] MFA Rules not processed for user `8020db56bca04d61b5a6e40384c25715`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:13.590473 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-d972fcf9-8f1d-4cfa-b962-3447ae904a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:13.596733 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-617167af-49c5-4210-8780-cce13dd26753 admin admin] The token cache is being invalidate because project ed187feab3f94be38411d79b7600d191 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:13.596898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-617167af-49c5-4210-8780-cce13dd26753 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:13.602723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-617167af-49c5-4210-8780-cce13dd26753 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:13.614662 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 852/1707] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:13 2022] DELETE /identity/v3/projects/ed187feab3f94be38411d79b7600d191 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:13.615586 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617167af-49c5-4210-8780-cce13dd26753 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:13.617392 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-617167af-49c5-4210-8780-cce13dd26753 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:13.807154 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 856/1708] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:13 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:13.813607 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b64b4-838c-423f-ab49-9ee9824095de None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:13.814061 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b64b4-838c-423f-ab49-9ee9824095de None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:13.814659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b64b4-838c-423f-ab49-9ee9824095de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:13.840443 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617167af-49c5-4210-8780-cce13dd26753 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:13.857498 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa413f-1a5c-4073-9473-a62e67b44682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:13.858150 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa413f-1a5c-4073-9473-a62e67b44682 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:13.858465 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa413f-1a5c-4073-9473-a62e67b44682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:13.858740 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa413f-1a5c-4073-9473-a62e67b44682 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:13.859450 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa413f-1a5c-4073-9473-a62e67b44682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:13.863333 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa413f-1a5c-4073-9473-a62e67b44682 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:13.889603 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-df9b64b4-838c-423f-ab49-9ee9824095de None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:13.892181 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-df9b64b4-838c-423f-ab49-9ee9824095de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:13.899102 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 853/1709] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:13 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:13.903503 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa413f-1a5c-4073-9473-a62e67b44682 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:13.907199 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-4cfa413f-1a5c-4073-9473-a62e67b44682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:13.934088 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 857/1710] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:13.936518 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa413f-1a5c-4073-9473-a62e67b44682 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:13.937632 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f341134-52bb-4ac3-8e30-7ba29ad1d939 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:13.941293 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f341134-52bb-4ac3-8e30-7ba29ad1d939 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:13.943592 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f341134-52bb-4ac3-8e30-7ba29ad1d939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:13.950309 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db27e4-ea2d-4cee-813b-6bc4ca0b19a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:13.950987 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db27e4-ea2d-4cee-813b-6bc4ca0b19a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:13.951268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db27e4-ea2d-4cee-813b-6bc4ca0b19a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:13.951545 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db27e4-ea2d-4cee-813b-6bc4ca0b19a2 admin admin] PATH_INFO: `/v3/users/8e7d5f3113c54e328e02a25d7af63964` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:13.957159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82db27e4-ea2d-4cee-813b-6bc4ca0b19a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e7d5f3113c54e328e02a25d7af63964)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:13.962443 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82db27e4-ea2d-4cee-813b-6bc4ca0b19a2 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:14.004474 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-1f341134-52bb-4ac3-8e30-7ba29ad1d939 None None] MFA Rules not processed for user `8020db56bca04d61b5a6e40384c25715`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:14.007898 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-1f341134-52bb-4ac3-8e30-7ba29ad1d939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:14.038190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 858/1711] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:13 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:14.039204 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f341134-52bb-4ac3-8e30-7ba29ad1d939 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:14.040593 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-1f341134-52bb-4ac3-8e30-7ba29ad1d939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:14.073695 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:14.081779 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-82db27e4-ea2d-4cee-813b-6bc4ca0b19a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e7d5f3113c54e328e02a25d7af63964'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:14.090231 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:14.099370 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-82db27e4-ea2d-4cee-813b-6bc4ca0b19a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e7d5f3113c54e328e02a25d7af63964'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:14.100527 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-82db27e4-ea2d-4cee-813b-6bc4ca0b19a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e7d5f3113c54e328e02a25d7af63964'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:14.133637 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-82db27e4-ea2d-4cee-813b-6bc4ca0b19a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e7d5f3113c54e328e02a25d7af63964'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:14.134595 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-82db27e4-ea2d-4cee-813b-6bc4ca0b19a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e7d5f3113c54e328e02a25d7af63964'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:14.165769 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 854/1712] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:13 2022] DELETE /identity/v3/users/8e7d5f3113c54e328e02a25d7af63964 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:14.166681 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db27e4-ea2d-4cee-813b-6bc4ca0b19a2 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:14.176202 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-82db27e4-ea2d-4cee-813b-6bc4ca0b19a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:14.266534 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f341134-52bb-4ac3-8e30-7ba29ad1d939 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:14.280906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db27e4-ea2d-4cee-813b-6bc4ca0b19a2 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:14.288690 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9950cb57-b59f-40e8-a367-3cfc4a09221b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:14.289840 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950cb57-b59f-40e8-a367-3cfc4a09221b admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:14.290272 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950cb57-b59f-40e8-a367-3cfc4a09221b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:14.290552 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950cb57-b59f-40e8-a367-3cfc4a09221b admin admin] PATH_INFO: `/v3/projects/3aaedec85cd54cfcbe5da73bc3290127/users/88e5060d2b43426a9c7cb0b8df042399/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:14.315027 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9950cb57-b59f-40e8-a367-3cfc4a09221b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3aaedec85cd54cfcbe5da73bc3290127, user_id=88e5060d2b43426a9c7cb0b8df042399, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:14.320926 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9950cb57-b59f-40e8-a367-3cfc4a09221b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:14.355575 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 855/1713] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:14 2022] PUT /identity/v3/projects/3aaedec85cd54cfcbe5da73bc3290127/users/88e5060d2b43426a9c7cb0b8df042399/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:14.358172 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9daa1b7-87a0-4cc9-87a3-102961e72981 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:14.358532 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9daa1b7-87a0-4cc9-87a3-102961e72981 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:14.358811 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9daa1b7-87a0-4cc9-87a3-102961e72981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:14.408395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5fa55-6661-4a3d-ae0e-ac560686aa61 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8137f6b2ced040bd9752239c7e987f03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:14.409098 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5fa55-6661-4a3d-ae0e-ac560686aa61 service swift] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:14.409383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5fa55-6661-4a3d-ae0e-ac560686aa61 service swift] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:14.409656 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5fa55-6661-4a3d-ae0e-ac560686aa61 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:14.414381 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-eff5fa55-6661-4a3d-ae0e-ac560686aa61 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:14.415395 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-f9daa1b7-87a0-4cc9-87a3-102961e72981 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:14.416888 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f9daa1b7-87a0-4cc9-87a3-102961e72981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:14.533147 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff5fa55-6661-4a3d-ae0e-ac560686aa61 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:14.535851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff5fa55-6661-4a3d-ae0e-ac560686aa61 service swift] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:14.550570 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 859/1714] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 1 16:34:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:34:14.551380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5fa55-6661-4a3d-ae0e-ac560686aa61 service swift] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:14.552985 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-eff5fa55-6661-4a3d-ae0e-ac560686aa61 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:14.580770 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 856/1715] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:14.580770 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9daa1b7-87a0-4cc9-87a3-102961e72981 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:14.600595 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9daa1b7-87a0-4cc9-87a3-102961e72981 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:14.610652 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1472d8-30c6-424f-9662-1e4b8a8285c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:14.611469 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1472d8-30c6-424f-9662-1e4b8a8285c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:14.611754 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1472d8-30c6-424f-9662-1e4b8a8285c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:14.612090 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1472d8-30c6-424f-9662-1e4b8a8285c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:14.612939 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1472d8-30c6-424f-9662-1e4b8a8285c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:14.618916 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1472d8-30c6-424f-9662-1e4b8a8285c5 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:14.622166 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:14.678054 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 857/1716] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:14.687117 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73fbbd-7e63-47c3-bc0a-7a10dd79a0e9 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:14.687523 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73fbbd-7e63-47c3-bc0a-7a10dd79a0e9 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:14.688224 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73fbbd-7e63-47c3-bc0a-7a10dd79a0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:14.736962 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5fa55-6661-4a3d-ae0e-ac560686aa61 service swift] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:14.798789 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-bf73fbbd-7e63-47c3-bc0a-7a10dd79a0e9 None None] MFA Rules not processed for user `8388303f9098497bb559b3bcb59fab52`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:14.800131 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-bf73fbbd-7e63-47c3-bc0a-7a10dd79a0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:14.936148 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590c629-8910-4ff5-9412-feba87d4482b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:14.936966 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590c629-8910-4ff5-9412-feba87d4482b service nova] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:14.937379 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590c629-8910-4ff5-9412-feba87d4482b service nova] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:14.937765 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590c629-8910-4ff5-9412-feba87d4482b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:14.940443 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-d590c629-8910-4ff5-9412-feba87d4482b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:15.035224 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:15.134196 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 858/1717] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:15.137638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73fbbd-7e63-47c3-bc0a-7a10dd79a0e9 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:15.152944 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d590c629-8910-4ff5-9412-feba87d4482b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:15.159029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d590c629-8910-4ff5-9412-feba87d4482b service nova] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:15.178376 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73fbbd-7e63-47c3-bc0a-7a10dd79a0e9 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:15.192124 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 860/1718] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:14 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:15.194815 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590c629-8910-4ff5-9412-feba87d4482b service nova] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:15.199993 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96751cb-35f1-44da-9808-f91558c8c795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:15.217343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96751cb-35f1-44da-9808-f91558c8c795 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:15.217461 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96751cb-35f1-44da-9808-f91558c8c795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:15.217808 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96751cb-35f1-44da-9808-f91558c8c795 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:15.221617 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96751cb-35f1-44da-9808-f91558c8c795 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:34:15.222028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96751cb-35f1-44da-9808-f91558c8c795 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:15.223729 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96751cb-35f1-44da-9808-f91558c8c795 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:15.240126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590c629-8910-4ff5-9412-feba87d4482b service nova] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:15.255183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16593cf6-7ab6-4f6c-a68a-767364a2a569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:15.258523 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16593cf6-7ab6-4f6c-a68a-767364a2a569 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:15.259186 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16593cf6-7ab6-4f6c-a68a-767364a2a569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:15.260303 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16593cf6-7ab6-4f6c-a68a-767364a2a569 admin admin] PATH_INFO: `/v3/projects/3aaedec85cd54cfcbe5da73bc3290127/users/88e5060d2b43426a9c7cb0b8df042399/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:15.267318 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16593cf6-7ab6-4f6c-a68a-767364a2a569 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3aaedec85cd54cfcbe5da73bc3290127, user_id=88e5060d2b43426a9c7cb0b8df042399, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:15.279516 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16593cf6-7ab6-4f6c-a68a-767364a2a569 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:15.307481 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 859/1719] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:34:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:34:15.309839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96751cb-35f1-44da-9808-f91558c8c795 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:15.331471 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 861/1720] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:15 2022] PUT /identity/v3/projects/3aaedec85cd54cfcbe5da73bc3290127/users/88e5060d2b43426a9c7cb0b8df042399/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:15.343775 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676a347-e938-4b4c-8a4f-1d318f6ed346 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:15.352888 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96751cb-35f1-44da-9808-f91558c8c795 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:15.353198 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676a347-e938-4b4c-8a4f-1d318f6ed346 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:15.353599 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676a347-e938-4b4c-8a4f-1d318f6ed346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:15.483744 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-1676a347-e938-4b4c-8a4f-1d318f6ed346 None None] MFA Rules not processed for user `88e5060d2b43426a9c7cb0b8df042399`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:15.492055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-1676a347-e938-4b4c-8a4f-1d318f6ed346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:15.640229 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316fd13a-b247-484c-b046-de00171768e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:15.642488 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fd13a-b247-484c-b046-de00171768e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:15.642790 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fd13a-b247-484c-b046-de00171768e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:15.643149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fd13a-b247-484c-b046-de00171768e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:15.646470 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316fd13a-b247-484c-b046-de00171768e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:15.648618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316fd13a-b247-484c-b046-de00171768e5 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:15.696813 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:15.795467 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 860/1721] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:15 2022] POST /identity/v3/projects => generated 357 bytes in 486 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:15.796950 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316fd13a-b247-484c-b046-de00171768e5 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:15.802921 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-316fd13a-b247-484c-b046-de00171768e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:15.822825 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 862/1722] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:15 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 478 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:15.823042 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5a8cf-dde1-4df8-be54-2c2bb6c1eae0 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:15.823282 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5a8cf-dde1-4df8-be54-2c2bb6c1eae0 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:15.823571 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5a8cf-dde1-4df8-be54-2c2bb6c1eae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:15.882007 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-ada5a8cf-dde1-4df8-be54-2c2bb6c1eae0 None None] MFA Rules not processed for user `ffa0f2160a934906b59e6f845edb0776`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:15.883729 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-ada5a8cf-dde1-4df8-be54-2c2bb6c1eae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:16.009551 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316fd13a-b247-484c-b046-de00171768e5 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:16.028326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3060092-c25c-4d14-afd5-d68d77ffc3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:16.029068 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3060092-c25c-4d14-afd5-d68d77ffc3af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:16.030753 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3060092-c25c-4d14-afd5-d68d77ffc3af admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:16.031125 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3060092-c25c-4d14-afd5-d68d77ffc3af admin admin] PATH_INFO: `/v3/projects/15bc0930beb341cfb9f5e8293c274efe` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:16.035282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3060092-c25c-4d14-afd5-d68d77ffc3af admin admin] RBAC: Authorizing `identity:delete_project(project_id=15bc0930beb341cfb9f5e8293c274efe)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:16.041447 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3060092-c25c-4d14-afd5-d68d77ffc3af admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:16.064918 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 863/1723] 10.222.0.77 () {58 vars in 1015 bytes} [Thu Sep 1 16:34:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:16.064918 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5a8cf-dde1-4df8-be54-2c2bb6c1eae0 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:16.100923 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5a8cf-dde1-4df8-be54-2c2bb6c1eae0 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:16.118452 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc65b7-4dd6-4419-ae1e-49b681b264e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:16.124559 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc65b7-4dd6-4419-ae1e-49b681b264e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:16.124859 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc65b7-4dd6-4419-ae1e-49b681b264e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:16.125142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc65b7-4dd6-4419-ae1e-49b681b264e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:16.126855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdc65b7-4dd6-4419-ae1e-49b681b264e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:16.136129 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdc65b7-4dd6-4419-ae1e-49b681b264e6 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:16.206996 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 864/1724] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:16 2022] POST /identity/v3/users => generated 279 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:16.213698 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc65b7-4dd6-4419-ae1e-49b681b264e6 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:16.219295 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-3fdc65b7-4dd6-4419-ae1e-49b681b264e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:16.222927 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c3060092-c25c-4d14-afd5-d68d77ffc3af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15bc0930beb341cfb9f5e8293c274efe', 'request_id': 'req-c3060092-c25c-4d14-afd5-d68d77ffc3af'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:16.245922 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c3060092-c25c-4d14-afd5-d68d77ffc3af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15bc0930beb341cfb9f5e8293c274efe', 'request_id': 'req-c3060092-c25c-4d14-afd5-d68d77ffc3af'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:16.279213 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c3060092-c25c-4d14-afd5-d68d77ffc3af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15bc0930beb341cfb9f5e8293c274efe', 'request_id': 'req-c3060092-c25c-4d14-afd5-d68d77ffc3af'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:16.280803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:16.301014 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c3060092-c25c-4d14-afd5-d68d77ffc3af admin admin] The token cache is being invalidate because project 15bc0930beb341cfb9f5e8293c274efe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:16.301324 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c3060092-c25c-4d14-afd5-d68d77ffc3af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:16.303246 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c3060092-c25c-4d14-afd5-d68d77ffc3af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:16.308521 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 861/1725] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:15 2022] DELETE /identity/v3/projects/15bc0930beb341cfb9f5e8293c274efe => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:16.309527 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3060092-c25c-4d14-afd5-d68d77ffc3af admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:16.317464 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c3060092-c25c-4d14-afd5-d68d77ffc3af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:16.428158 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc65b7-4dd6-4419-ae1e-49b681b264e6 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:16.485773 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3060092-c25c-4d14-afd5-d68d77ffc3af admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:16.498923 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21ab3f-77d5-4cc4-b718-78e855b22b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:16.499483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21ab3f-77d5-4cc4-b718-78e855b22b58 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:16.499770 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21ab3f-77d5-4cc4-b718-78e855b22b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:16.500191 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21ab3f-77d5-4cc4-b718-78e855b22b58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:16.500693 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21ab3f-77d5-4cc4-b718-78e855b22b58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:16.502350 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21ab3f-77d5-4cc4-b718-78e855b22b58 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:16.524199 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 862/1726] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:16 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:16.525804 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a306077-025f-4621-84e9-46606214d773 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:16.526229 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a306077-025f-4621-84e9-46606214d773 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:16.527015 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a306077-025f-4621-84e9-46606214d773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:16.546143 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55fbfc-b367-4fde-80e1-e44dc2df4576 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:16.546966 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55fbfc-b367-4fde-80e1-e44dc2df4576 service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:16.547372 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55fbfc-b367-4fde-80e1-e44dc2df4576 service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:16.548507 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55fbfc-b367-4fde-80e1-e44dc2df4576 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:16.550998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-7c55fbfc-b367-4fde-80e1-e44dc2df4576 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:16.574837 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-1a306077-025f-4621-84e9-46606214d773 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:16.576099 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-1a306077-025f-4621-84e9-46606214d773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:16.620482 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 863/1727] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:16.620482 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a306077-025f-4621-84e9-46606214d773 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:16.645438 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a306077-025f-4621-84e9-46606214d773 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:16.657720 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96a931-f800-4fdd-9e67-32b871265472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:16.661470 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96a931-f800-4fdd-9e67-32b871265472 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:16.661758 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96a931-f800-4fdd-9e67-32b871265472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:16.662033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96a931-f800-4fdd-9e67-32b871265472 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:16.662738 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d96a931-f800-4fdd-9e67-32b871265472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:16.666315 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d96a931-f800-4fdd-9e67-32b871265472 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:16.695891 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:16.706737 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 864/1728] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:16 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:16.711668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec37be-15ca-4f05-8baa-0673b2a8acab None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:16.712016 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec37be-15ca-4f05-8baa-0673b2a8acab None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:16.712232 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec37be-15ca-4f05-8baa-0673b2a8acab None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:16.717673 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 865/1729] 10.222.0.18 () {54 vars in 912 bytes} [Thu Sep 1 16:34:16 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:34:16.721683 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060de9e-e8cd-4fea-ab8e-e5902de8bf45 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:16.721982 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060de9e-e8cd-4fea-ab8e-e5902de8bf45 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:16.722352 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060de9e-e8cd-4fea-ab8e-e5902de8bf45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:16.755464 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c55fbfc-b367-4fde-80e1-e44dc2df4576 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:16.758005 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c55fbfc-b367-4fde-80e1-e44dc2df4576 service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:16.769445 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-0060de9e-e8cd-4fea-ab8e-e5902de8bf45 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:16.771295 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0060de9e-e8cd-4fea-ab8e-e5902de8bf45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:16.789377 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 866/1730] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:16.790988 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0060de9e-e8cd-4fea-ab8e-e5902de8bf45 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:16.798624 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 865/1731] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:16 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:16.800242 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160b58c-1bde-4e6d-9618-49ad414a08c6 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:16.805714 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160b58c-1bde-4e6d-9618-49ad414a08c6 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:16.806021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160b58c-1bde-4e6d-9618-49ad414a08c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:16.814913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0060de9e-e8cd-4fea-ab8e-e5902de8bf45 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:16.820899 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbd8fcb-1512-45f7-b8fe-188c96e4e472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:16.821586 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd8fcb-1512-45f7-b8fe-188c96e4e472 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:16.822289 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd8fcb-1512-45f7-b8fe-188c96e4e472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:16.822580 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd8fcb-1512-45f7-b8fe-188c96e4e472 admin admin] PATH_INFO: `/v3/projects/21a5fde5baa94acdace581161f41f356/users/e8f3a2285b9d481990a230b7c389ce70/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:16.847267 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbd8fcb-1512-45f7-b8fe-188c96e4e472 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21a5fde5baa94acdace581161f41f356, user_id=e8f3a2285b9d481990a230b7c389ce70, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:16.851431 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbd8fcb-1512-45f7-b8fe-188c96e4e472 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:16.872862 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-8160b58c-1bde-4e6d-9618-49ad414a08c6 None None] MFA Rules not processed for user `ffa0f2160a934906b59e6f845edb0776`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:16.884806 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-8160b58c-1bde-4e6d-9618-49ad414a08c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:16.926263 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 867/1732] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:16 2022] PUT /identity/v3/projects/21a5fde5baa94acdace581161f41f356/users/e8f3a2285b9d481990a230b7c389ce70/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:16.926263 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbd8fcb-1512-45f7-b8fe-188c96e4e472 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:17.169636 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 866/1733] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Sep 1 16:34:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:17.171157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4642e-0052-467e-b43d-916a13df3ffc None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:17.171494 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4642e-0052-467e-b43d-916a13df3ffc None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:17.171773 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4642e-0052-467e-b43d-916a13df3ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:17.222294 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbd8fcb-1512-45f7-b8fe-188c96e4e472 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:17.239639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e9aa6-a2a5-48c4-8834-30267bf8fa01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:17.240900 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e9aa6-a2a5-48c4-8834-30267bf8fa01 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:17.242370 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e9aa6-a2a5-48c4-8834-30267bf8fa01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:17.242868 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e9aa6-a2a5-48c4-8834-30267bf8fa01 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:17.245230 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e9aa6-a2a5-48c4-8834-30267bf8fa01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:34:17.245230 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e9aa6-a2a5-48c4-8834-30267bf8fa01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:17.246253 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e9aa6-a2a5-48c4-8834-30267bf8fa01 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:17.257827 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-8be4642e-0052-467e-b43d-916a13df3ffc None None] MFA Rules not processed for user `e8f3a2285b9d481990a230b7c389ce70`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:17.257827 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-8be4642e-0052-467e-b43d-916a13df3ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:17.288942 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 868/1734] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:34:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:34:17.294654 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e9aa6-a2a5-48c4-8834-30267bf8fa01 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:17.299712 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b11e9aa6-a2a5-48c4-8834-30267bf8fa01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:17.415103 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 867/1735] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:17 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:17.415995 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4642e-0052-467e-b43d-916a13df3ffc None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:17.417334 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-8be4642e-0052-467e-b43d-916a13df3ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:17.427957 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:17.432642 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4642e-0052-467e-b43d-916a13df3ffc None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:17.439478 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61230e2-3b08-4998-96f8-b53b061c19da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:17.440478 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61230e2-3b08-4998-96f8-b53b061c19da service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:17.440766 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61230e2-3b08-4998-96f8-b53b061c19da service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:17.441045 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61230e2-3b08-4998-96f8-b53b061c19da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:17.443264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-a61230e2-3b08-4998-96f8-b53b061c19da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:17.461069 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61230e2-3b08-4998-96f8-b53b061c19da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:17.478985 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61230e2-3b08-4998-96f8-b53b061c19da service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:17.492211 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e9aa6-a2a5-48c4-8834-30267bf8fa01 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:17.496991 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6aca61-f6cb-4139-b628-eef53352f8ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:17.498508 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6aca61-f6cb-4139-b628-eef53352f8ac service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:17.498796 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6aca61-f6cb-4139-b628-eef53352f8ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:17.499114 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6aca61-f6cb-4139-b628-eef53352f8ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:17.503075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ba6aca61-f6cb-4139-b628-eef53352f8ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:17.506868 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 868/1736] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:17.509079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc3003-91cf-4c6c-8d5f-f58f7386df42 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:17.509375 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc3003-91cf-4c6c-8d5f-f58f7386df42 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:17.512166 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc3003-91cf-4c6c-8d5f-f58f7386df42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:17.570152 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-55cc3003-91cf-4c6c-8d5f-f58f7386df42 None None] MFA Rules not processed for user `73624e79ccdd4782beddf663bb6d02e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:17.571542 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-55cc3003-91cf-4c6c-8d5f-f58f7386df42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:17.745843 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6aca61-f6cb-4139-b628-eef53352f8ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:17.749538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6aca61-f6cb-4139-b628-eef53352f8ac service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:17.777746 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 869/1737] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:17.789150 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 869/1738] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:17.790165 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6aca61-f6cb-4139-b628-eef53352f8ac service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:17.798420 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc3003-91cf-4c6c-8d5f-f58f7386df42 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:17.800977 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-55cc3003-91cf-4c6c-8d5f-f58f7386df42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:17.820118 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6aca61-f6cb-4139-b628-eef53352f8ac service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:17.836109 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b16e40-0be5-4f47-8a14-070f24c9b057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:17.844251 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc3003-91cf-4c6c-8d5f-f58f7386df42 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:17.846159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b16e40-0be5-4f47-8a14-070f24c9b057 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:17.846576 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b16e40-0be5-4f47-8a14-070f24c9b057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:17.846912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b16e40-0be5-4f47-8a14-070f24c9b057 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:17.847741 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b16e40-0be5-4f47-8a14-070f24c9b057 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:17.851072 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dded88-dd40-4363-8e48-22c2d317ee9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:17.851409 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b16e40-0be5-4f47-8a14-070f24c9b057 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:17.856931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dded88-dd40-4363-8e48-22c2d317ee9a service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:17.857265 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dded88-dd40-4363-8e48-22c2d317ee9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:17.857575 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dded88-dd40-4363-8e48-22c2d317ee9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:17.875763 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:17.881114 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dded88-dd40-4363-8e48-22c2d317ee9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:17.888280 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dded88-dd40-4363-8e48-22c2d317ee9a service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:17.904310 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 870/1739] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:17.906447 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dded88-dd40-4363-8e48-22c2d317ee9a service neutron] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:17.941171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 870/1740] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:17 2022] POST /identity/v3/projects => generated 365 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:17.941343 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dded88-dd40-4363-8e48-22c2d317ee9a service neutron] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:17.941438 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38dfbab-f64b-4672-9d88-2880472793f0 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:17.941438 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38dfbab-f64b-4672-9d88-2880472793f0 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:17.941438 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38dfbab-f64b-4672-9d88-2880472793f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:18.006948 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-b38dfbab-f64b-4672-9d88-2880472793f0 None None] MFA Rules not processed for user `8020db56bca04d61b5a6e40384c25715`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:18.007863 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b38dfbab-f64b-4672-9d88-2880472793f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:18.211007 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a3939-551b-49ed-a208-c058d3179cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:18.211636 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a3939-551b-49ed-a208-c058d3179cf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:18.211921 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a3939-551b-49ed-a208-c058d3179cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:18.212204 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a3939-551b-49ed-a208-c058d3179cf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:18.212972 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151a3939-551b-49ed-a208-c058d3179cf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:18.215525 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151a3939-551b-49ed-a208-c058d3179cf0 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:18.291628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 871/1741] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:17 2022] POST /identity/v3/projects => generated 357 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:18.294012 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a3939-551b-49ed-a208-c058d3179cf0 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:18.305528 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 871/1742] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:18.313385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38dfbab-f64b-4672-9d88-2880472793f0 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:18.341533 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38dfbab-f64b-4672-9d88-2880472793f0 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:18.566275 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a3939-551b-49ed-a208-c058d3179cf0 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:18.583142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c2f84a-e18d-4719-b215-7cb6babb4560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:18.583788 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2f84a-e18d-4719-b215-7cb6babb4560 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:18.584091 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2f84a-e18d-4719-b215-7cb6babb4560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:18.584366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2f84a-e18d-4719-b215-7cb6babb4560 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:18.585016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c2f84a-e18d-4719-b215-7cb6babb4560 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:18.587172 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c2f84a-e18d-4719-b215-7cb6babb4560 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:18.607329 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df9582-07f4-46d2-b6b4-9b888d4147c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:18.608033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df9582-07f4-46d2-b6b4-9b888d4147c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:18.608320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df9582-07f4-46d2-b6b4-9b888d4147c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:18.608609 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df9582-07f4-46d2-b6b4-9b888d4147c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:18.609248 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df9582-07f4-46d2-b6b4-9b888d4147c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:18.615277 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df9582-07f4-46d2-b6b4-9b888d4147c5 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:18.645013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 872/1743] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:18 2022] POST /identity/v3/users => generated 283 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:18.647204 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c2f84a-e18d-4719-b215-7cb6babb4560 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:18.669393 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:18.672839 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c2f84a-e18d-4719-b215-7cb6babb4560 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:18.686047 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 872/1744] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:18 2022] POST /identity/v3/users => generated 285 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:18.687182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df9582-07f4-46d2-b6b4-9b888d4147c5 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:18.693418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6811a8-5d67-45e8-8e60-6b352613e741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:18.697765 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6811a8-5d67-45e8-8e60-6b352613e741 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:18.698426 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6811a8-5d67-45e8-8e60-6b352613e741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:18.699593 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6811a8-5d67-45e8-8e60-6b352613e741 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:18.700510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6811a8-5d67-45e8-8e60-6b352613e741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:18.702944 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6811a8-5d67-45e8-8e60-6b352613e741 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:18.716190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df9582-07f4-46d2-b6b4-9b888d4147c5 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:18.730086 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 873/1745] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:18.733431 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6811a8-5d67-45e8-8e60-6b352613e741 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:18.745008 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d6697-7610-401e-b83f-7575b3d8b4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:18.750605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d6697-7610-401e-b83f-7575b3d8b4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:18.751006 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d6697-7610-401e-b83f-7575b3d8b4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:18.769602 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d6697-7610-401e-b83f-7575b3d8b4d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:18.771574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d6697-7610-401e-b83f-7575b3d8b4d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:18.779416 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d6697-7610-401e-b83f-7575b3d8b4d3 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:18.783913 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6811a8-5d67-45e8-8e60-6b352613e741 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:18.798807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183bcf35-aacc-4c8c-b655-c1ea2a09097d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:18.799528 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bcf35-aacc-4c8c-b655-c1ea2a09097d admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:18.799891 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bcf35-aacc-4c8c-b655-c1ea2a09097d admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:18.800241 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bcf35-aacc-4c8c-b655-c1ea2a09097d admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:18.800965 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183bcf35-aacc-4c8c-b655-c1ea2a09097d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:18.802792 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183bcf35-aacc-4c8c-b655-c1ea2a09097d admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:18.829429 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 873/1746] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:18.832718 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d6697-7610-401e-b83f-7575b3d8b4d3 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:18.836995 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-de9d6697-7610-401e-b83f-7575b3d8b4d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:18.859610 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 874/1747] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:18.861012 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183bcf35-aacc-4c8c-b655-c1ea2a09097d admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:18.888910 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183bcf35-aacc-4c8c-b655-c1ea2a09097d admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:18.920726 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af4b08-3cdb-431f-ba37-37960f84ad9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:18.922294 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af4b08-3cdb-431f-ba37-37960f84ad9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:18.923615 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af4b08-3cdb-431f-ba37-37960f84ad9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:18.923615 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af4b08-3cdb-431f-ba37-37960f84ad9e admin admin] PATH_INFO: `/v3/projects/a10702e556664b14bd6cb7be929859f6/users/65114cc4ffba44438bbd2843e86184f8/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:18.964590 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af4b08-3cdb-431f-ba37-37960f84ad9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a10702e556664b14bd6cb7be929859f6, user_id=65114cc4ffba44438bbd2843e86184f8, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:18.967775 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af4b08-3cdb-431f-ba37-37960f84ad9e admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:18.997695 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 875/1748] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:18 2022] PUT /identity/v3/projects/a10702e556664b14bd6cb7be929859f6/users/65114cc4ffba44438bbd2843e86184f8/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:19.007685 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af4b08-3cdb-431f-ba37-37960f84ad9e admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:19.127403 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d6697-7610-401e-b83f-7575b3d8b4d3 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:19.156203 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:19.320026 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af4b08-3cdb-431f-ba37-37960f84ad9e admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:19.335695 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f8945-c74a-41e3-805f-af28419c7b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:19.337031 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f8945-c74a-41e3-805f-af28419c7b26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:19.337630 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f8945-c74a-41e3-805f-af28419c7b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:19.338135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f8945-c74a-41e3-805f-af28419c7b26 admin admin] PATH_INFO: `/v3/projects/758057d6037e4765a2522c393905f50a/users/94d1c2c11c5c49cfa6db06a8c76b1635/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:19.390594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f8945-c74a-41e3-805f-af28419c7b26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=758057d6037e4765a2522c393905f50a, user_id=94d1c2c11c5c49cfa6db06a8c76b1635, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:19.397264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f8945-c74a-41e3-805f-af28419c7b26 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:19.417961 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe8efc-7349-4cff-8818-602b32aa0094 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:19.418817 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe8efc-7349-4cff-8818-602b32aa0094 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:19.419212 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe8efc-7349-4cff-8818-602b32aa0094 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:19.419599 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe8efc-7349-4cff-8818-602b32aa0094 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:19.426825 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-21fe8efc-7349-4cff-8818-602b32aa0094 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:19.438737 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 876/1749] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:18 2022] PUT /identity/v3/projects/758057d6037e4765a2522c393905f50a/users/94d1c2c11c5c49cfa6db06a8c76b1635/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:19.440467 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f8945-c74a-41e3-805f-af28419c7b26 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:19.528251 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f8945-c74a-41e3-805f-af28419c7b26 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:19.533987 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebdf034-01f6-4f29-a388-fff8c515ca4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:19.534714 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdf034-01f6-4f29-a388-fff8c515ca4d admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:19.534998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdf034-01f6-4f29-a388-fff8c515ca4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:19.535275 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdf034-01f6-4f29-a388-fff8c515ca4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:19.543758 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebdf034-01f6-4f29-a388-fff8c515ca4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:19.545332 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebdf034-01f6-4f29-a388-fff8c515ca4d admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:19.568843 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 877/1750] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:19.571811 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb1b12b-7b85-4ca3-8ef6-4dd256b41045 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:19.571811 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb1b12b-7b85-4ca3-8ef6-4dd256b41045 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:19.571811 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb1b12b-7b85-4ca3-8ef6-4dd256b41045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:19.673800 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-bcb1b12b-7b85-4ca3-8ef6-4dd256b41045 None None] MFA Rules not processed for user `94d1c2c11c5c49cfa6db06a8c76b1635`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:19.677132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-bcb1b12b-7b85-4ca3-8ef6-4dd256b41045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:19.680159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe8efc-7349-4cff-8818-602b32aa0094 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:19.682051 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe8efc-7349-4cff-8818-602b32aa0094 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:19.694691 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 874/1751] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:18 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 864 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:19.695945 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe8efc-7349-4cff-8818-602b32aa0094 service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:19.762985 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe8efc-7349-4cff-8818-602b32aa0094 service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:19.794715 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b02dc0-1a67-49a3-8f2b-f6906b5832ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:19.795523 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b02dc0-1a67-49a3-8f2b-f6906b5832ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:19.795939 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b02dc0-1a67-49a3-8f2b-f6906b5832ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:19.796443 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b02dc0-1a67-49a3-8f2b-f6906b5832ae admin admin] PATH_INFO: `/v3/projects/a10702e556664b14bd6cb7be929859f6/users/65114cc4ffba44438bbd2843e86184f8/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:19.826177 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b02dc0-1a67-49a3-8f2b-f6906b5832ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=a10702e556664b14bd6cb7be929859f6, user_id=65114cc4ffba44438bbd2843e86184f8, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:19.835491 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b02dc0-1a67-49a3-8f2b-f6906b5832ae admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:19.890719 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 878/1753] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:19.894115 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78b9cb-7794-4772-b018-916ec6e5116e None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:19.899584 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78b9cb-7794-4772-b018-916ec6e5116e None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:19.899839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 875/1753] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:19 2022] PUT /identity/v3/projects/a10702e556664b14bd6cb7be929859f6/users/65114cc4ffba44438bbd2843e86184f8/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:19.900313 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78b9cb-7794-4772-b018-916ec6e5116e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:19.903455 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b02dc0-1a67-49a3-8f2b-f6906b5832ae admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:19.992374 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:20.028355 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-ff78b9cb-7794-4772-b018-916ec6e5116e None None] MFA Rules not processed for user `65114cc4ffba44438bbd2843e86184f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:20.033785 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-ff78b9cb-7794-4772-b018-916ec6e5116e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:20.126129 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b02dc0-1a67-49a3-8f2b-f6906b5832ae admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:20.141115 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83449a-8c3b-4d88-9971-e29158da2121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:20.142682 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83449a-8c3b-4d88-9971-e29158da2121 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:20.143144 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83449a-8c3b-4d88-9971-e29158da2121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:20.143628 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83449a-8c3b-4d88-9971-e29158da2121 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:20.144389 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e83449a-8c3b-4d88-9971-e29158da2121 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:20.147096 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e83449a-8c3b-4d88-9971-e29158da2121 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:20.150289 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:20.243328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 876/1754] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:19 2022] POST /identity/v3/projects => generated 363 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:20.247031 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83449a-8c3b-4d88-9971-e29158da2121 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:20.336637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 879/1755] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:19 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 445 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:20.372923 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff78b9cb-7794-4772-b018-916ec6e5116e None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:20.491716 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83449a-8c3b-4d88-9971-e29158da2121 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:20.509153 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a3629-d9be-45ec-8fed-aafd9a4e33db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:20.509856 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a3629-d9be-45ec-8fed-aafd9a4e33db admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:20.510150 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a3629-d9be-45ec-8fed-aafd9a4e33db admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:20.510523 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a3629-d9be-45ec-8fed-aafd9a4e33db admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:20.511891 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a3629-d9be-45ec-8fed-aafd9a4e33db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:20.514060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a3629-d9be-45ec-8fed-aafd9a4e33db admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:20.552612 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff78b9cb-7794-4772-b018-916ec6e5116e None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:20.574602 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 877/1756] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:20 2022] POST /identity/v3/users => generated 288 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:20.575579 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a3629-d9be-45ec-8fed-aafd9a4e33db admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:20.609872 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a3629-d9be-45ec-8fed-aafd9a4e33db admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:20.629701 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034fcfbe-6ae8-427d-bb2b-48b61d33d5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:20.630431 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034fcfbe-6ae8-427d-bb2b-48b61d33d5fd admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:20.630720 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034fcfbe-6ae8-427d-bb2b-48b61d33d5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:20.631001 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034fcfbe-6ae8-427d-bb2b-48b61d33d5fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:20.640266 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034fcfbe-6ae8-427d-bb2b-48b61d33d5fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:20.648159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034fcfbe-6ae8-427d-bb2b-48b61d33d5fd admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:20.693874 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 878/1757] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:20.694848 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034fcfbe-6ae8-427d-bb2b-48b61d33d5fd admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:20.725632 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034fcfbe-6ae8-427d-bb2b-48b61d33d5fd admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:20.742892 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd818e6-94d2-401d-bb1c-f5292f343707 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:20.743557 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd818e6-94d2-401d-bb1c-f5292f343707 service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:20.744927 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd818e6-94d2-401d-bb1c-f5292f343707 service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:20.745211 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd818e6-94d2-401d-bb1c-f5292f343707 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:20.746492 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eee44e-8e1f-435d-8150-fcbe677913ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:20.747175 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eee44e-8e1f-435d-8150-fcbe677913ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:20.747445 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eee44e-8e1f-435d-8150-fcbe677913ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:20.747814 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eee44e-8e1f-435d-8150-fcbe677913ec admin admin] PATH_INFO: `/v3/projects/f9a21b664d254044890e6f7f92d2593a/users/a2efd5f6dc104d089802e376e90916d7/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:20.827666 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eee44e-8e1f-435d-8150-fcbe677913ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9a21b664d254044890e6f7f92d2593a, user_id=a2efd5f6dc104d089802e376e90916d7, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:20.827666 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eee44e-8e1f-435d-8150-fcbe677913ec admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:20.868375 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 879/1758] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:20 2022] PUT /identity/v3/projects/f9a21b664d254044890e6f7f92d2593a/users/a2efd5f6dc104d089802e376e90916d7/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:20.869524 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eee44e-8e1f-435d-8150-fcbe677913ec admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:20.958771 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eee44e-8e1f-435d-8150-fcbe677913ec admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:20.966070 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ed111-12e0-4507-aadc-1e34b3a83726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:20.966794 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ed111-12e0-4507-aadc-1e34b3a83726 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:20.967080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ed111-12e0-4507-aadc-1e34b3a83726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:20.967356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ed111-12e0-4507-aadc-1e34b3a83726 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:20.968068 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ed111-12e0-4507-aadc-1e34b3a83726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:20.969980 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ed111-12e0-4507-aadc-1e34b3a83726 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:20.987544 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 880/1759] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:20.988720 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ed111-12e0-4507-aadc-1e34b3a83726 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:21.003316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ed111-12e0-4507-aadc-1e34b3a83726 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:21.008818 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e4c48-e6c2-42bf-970f-0d865351b68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:21.009655 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e4c48-e6c2-42bf-970f-0d865351b68d admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:21.010278 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e4c48-e6c2-42bf-970f-0d865351b68d admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:21.010278 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e4c48-e6c2-42bf-970f-0d865351b68d admin admin] PATH_INFO: `/v3/projects/f9a21b664d254044890e6f7f92d2593a/users/a2efd5f6dc104d089802e376e90916d7/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:21.013302 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0e4c48-e6c2-42bf-970f-0d865351b68d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9a21b664d254044890e6f7f92d2593a, user_id=a2efd5f6dc104d089802e376e90916d7, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:21.016282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0e4c48-e6c2-42bf-970f-0d865351b68d admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:21.039129 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 881/1760] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:20 2022] PUT /identity/v3/projects/f9a21b664d254044890e6f7f92d2593a/users/a2efd5f6dc104d089802e376e90916d7/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:21.040644 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c017e0a-3f85-4b31-8332-40d20f23d7d3 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:21.040945 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c017e0a-3f85-4b31-8332-40d20f23d7d3 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:21.041291 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c017e0a-3f85-4b31-8332-40d20f23d7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:21.048868 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:21.079904 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-5c017e0a-3f85-4b31-8332-40d20f23d7d3 None None] MFA Rules not processed for user `a2efd5f6dc104d089802e376e90916d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:21.081339 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-5c017e0a-3f85-4b31-8332-40d20f23d7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:21.094693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd818e6-94d2-401d-bb1c-f5292f343707 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:21.099303 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd818e6-94d2-401d-bb1c-f5292f343707 service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:21.140087 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 880/1761] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:20 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 767 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:21.177385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 882/1762] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:21 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:21.193531 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd818e6-94d2-401d-bb1c-f5292f343707 service cinder] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:21.208525 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd818e6-94d2-401d-bb1c-f5292f343707 service cinder] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:21.243198 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:21.405067 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40777a2-270e-436f-ac87-d778263e9a50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:21.405777 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40777a2-270e-436f-ac87-d778263e9a50 service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:21.412124 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40777a2-270e-436f-ac87-d778263e9a50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:21.412413 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40777a2-270e-436f-ac87-d778263e9a50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:21.457154 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40777a2-270e-436f-ac87-d778263e9a50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:21.461752 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40777a2-270e-436f-ac87-d778263e9a50 service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:21.465973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1e757-0ea4-462c-8481-2bb1d255d4e5 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:21.466291 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1e757-0ea4-462c-8481-2bb1d255d4e5 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:21.466588 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1e757-0ea4-462c-8481-2bb1d255d4e5 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:21.468226 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 883/1763] 10.222.0.18 () {54 vars in 921 bytes} [Thu Sep 1 16:34:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:34:21.495748 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c0404-6d2d-4fc2-a574-e1955cb123e0 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:21.496557 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c0404-6d2d-4fc2-a574-e1955cb123e0 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:21.496557 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c0404-6d2d-4fc2-a574-e1955cb123e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:21.499884 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 881/1764] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:21 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:21.554648 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-3d1c0404-6d2d-4fc2-a574-e1955cb123e0 None None] MFA Rules not processed for user `278e6d6aed414c758fe3e9f4df6bf81e`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:21.558570 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3d1c0404-6d2d-4fc2-a574-e1955cb123e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:21.561684 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40777a2-270e-436f-ac87-d778263e9a50 service cinder] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:21.571724 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40777a2-270e-436f-ac87-d778263e9a50 service cinder] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:21.595010 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d7f2a-ff0a-42ba-a4f2-6f962e38e060 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:21.596447 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d7f2a-ff0a-42ba-a4f2-6f962e38e060 service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:21.596740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d7f2a-ff0a-42ba-a4f2-6f962e38e060 service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:21.597068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d7f2a-ff0a-42ba-a4f2-6f962e38e060 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:21.723799 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d7f2a-ff0a-42ba-a4f2-6f962e38e060 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:21.725459 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d7f2a-ff0a-42ba-a4f2-6f962e38e060 service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:21.751695 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 882/1765] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:21 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:34:21.784703 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 884/1766] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Sep 1 16:34:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:34:22.114276 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d7f2a-ff0a-42ba-a4f2-6f962e38e060 service cinder] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:22.128726 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d7f2a-ff0a-42ba-a4f2-6f962e38e060 service cinder] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:22.134857 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e892f4fc-077c-458c-abb9-5019193a0897 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:22.135697 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892f4fc-077c-458c-abb9-5019193a0897 service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:22.136270 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892f4fc-077c-458c-abb9-5019193a0897 service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:22.136659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892f4fc-077c-458c-abb9-5019193a0897 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:22.292911 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e892f4fc-077c-458c-abb9-5019193a0897 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:22.294468 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e892f4fc-077c-458c-abb9-5019193a0897 service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:22.310070 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 883/1767] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:22.384087 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c0404-6d2d-4fc2-a574-e1955cb123e0 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:22.385916 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3d1c0404-6d2d-4fc2-a574-e1955cb123e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:22.408692 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c0404-6d2d-4fc2-a574-e1955cb123e0 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:22.422360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc322f15-40b3-4b80-a592-61bc0f097fef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:22.423258 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc322f15-40b3-4b80-a592-61bc0f097fef service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:22.423689 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc322f15-40b3-4b80-a592-61bc0f097fef service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:22.424167 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc322f15-40b3-4b80-a592-61bc0f097fef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:22.438540 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc322f15-40b3-4b80-a592-61bc0f097fef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:22.442534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc322f15-40b3-4b80-a592-61bc0f097fef service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:22.488121 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 885/1768] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:28.759519 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e892f4fc-077c-458c-abb9-5019193a0897 service cinder] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:28.881189 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e892f4fc-077c-458c-abb9-5019193a0897 service cinder] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:28.886565 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9347d-7be6-4bf8-8c68-95f9f139383c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:28.887219 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9347d-7be6-4bf8-8c68-95f9f139383c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:28.887501 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9347d-7be6-4bf8-8c68-95f9f139383c admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:28.887772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9347d-7be6-4bf8-8c68-95f9f139383c admin admin] PATH_INFO: `/v3/users/94d1c2c11c5c49cfa6db06a8c76b1635` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:28.890048 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c9347d-7be6-4bf8-8c68-95f9f139383c admin admin] RBAC: Authorizing `identity:delete_user(user_id=94d1c2c11c5c49cfa6db06a8c76b1635)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:28.891404 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c9347d-7be6-4bf8-8c68-95f9f139383c admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:28.897180 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:28.961895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-82c9347d-7be6-4bf8-8c68-95f9f139383c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94d1c2c11c5c49cfa6db06a8c76b1635'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:28.962898 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-82c9347d-7be6-4bf8-8c68-95f9f139383c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94d1c2c11c5c49cfa6db06a8c76b1635'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:28.974887 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-82c9347d-7be6-4bf8-8c68-95f9f139383c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94d1c2c11c5c49cfa6db06a8c76b1635'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:28.975648 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-82c9347d-7be6-4bf8-8c68-95f9f139383c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94d1c2c11c5c49cfa6db06a8c76b1635'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:29.005049 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-82c9347d-7be6-4bf8-8c68-95f9f139383c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94d1c2c11c5c49cfa6db06a8c76b1635'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:29.034537 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 884/1769] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:28 2022] DELETE /identity/v3/users/94d1c2c11c5c49cfa6db06a8c76b1635 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:29.041403 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f033f4-4e38-4cdf-8742-2f53e7fc3426 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:29.042543 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f033f4-4e38-4cdf-8742-2f53e7fc3426 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:29.043002 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f033f4-4e38-4cdf-8742-2f53e7fc3426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:29.093092 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-36f033f4-4e38-4cdf-8742-2f53e7fc3426 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:29.096644 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-36f033f4-4e38-4cdf-8742-2f53e7fc3426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:29.205362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 886/1770] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:29.215445 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9347d-7be6-4bf8-8c68-95f9f139383c admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:29.219108 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-82c9347d-7be6-4bf8-8c68-95f9f139383c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:29.326984 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9347d-7be6-4bf8-8c68-95f9f139383c admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:29.331430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae8e69-d0fd-4880-8806-4d447328d82f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:29.332068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae8e69-d0fd-4880-8806-4d447328d82f service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:29.332352 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae8e69-d0fd-4880-8806-4d447328d82f service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:29.332627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae8e69-d0fd-4880-8806-4d447328d82f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:29.343526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ae8e69-d0fd-4880-8806-4d447328d82f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:29.345391 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ae8e69-d0fd-4880-8806-4d447328d82f service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:29.362495 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 885/1771] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:29.728583 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f033f4-4e38-4cdf-8742-2f53e7fc3426 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:29.730412 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-36f033f4-4e38-4cdf-8742-2f53e7fc3426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:29.766881 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f033f4-4e38-4cdf-8742-2f53e7fc3426 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:29.775176 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb58066-92e6-4f5e-8152-bc0b3ac9168e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:29.775902 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb58066-92e6-4f5e-8152-bc0b3ac9168e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:29.776101 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb58066-92e6-4f5e-8152-bc0b3ac9168e admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:29.776488 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb58066-92e6-4f5e-8152-bc0b3ac9168e admin admin] PATH_INFO: `/v3/projects/758057d6037e4765a2522c393905f50a` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:29.778424 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb58066-92e6-4f5e-8152-bc0b3ac9168e admin admin] RBAC: Authorizing `identity:delete_project(project_id=758057d6037e4765a2522c393905f50a)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:29.781887 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb58066-92e6-4f5e-8152-bc0b3ac9168e admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:29.784330 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:29.851562 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-deb58066-92e6-4f5e-8152-bc0b3ac9168e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '758057d6037e4765a2522c393905f50a', 'request_id': 'req-deb58066-92e6-4f5e-8152-bc0b3ac9168e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:29.861098 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-deb58066-92e6-4f5e-8152-bc0b3ac9168e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '758057d6037e4765a2522c393905f50a', 'request_id': 'req-deb58066-92e6-4f5e-8152-bc0b3ac9168e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:29.872407 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-deb58066-92e6-4f5e-8152-bc0b3ac9168e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '758057d6037e4765a2522c393905f50a', 'request_id': 'req-deb58066-92e6-4f5e-8152-bc0b3ac9168e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:29.885422 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-deb58066-92e6-4f5e-8152-bc0b3ac9168e admin admin] The token cache is being invalidate because project 758057d6037e4765a2522c393905f50a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:29.885678 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-deb58066-92e6-4f5e-8152-bc0b3ac9168e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:29.886380 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-deb58066-92e6-4f5e-8152-bc0b3ac9168e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:29.888223 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 887/1772] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:29 2022] DELETE /identity/v3/projects/758057d6037e4765a2522c393905f50a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:29.892002 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae8e69-d0fd-4880-8806-4d447328d82f service neutron] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:29.902497 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae8e69-d0fd-4880-8806-4d447328d82f service neutron] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:29.969446 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa9376-667d-45cc-bd1e-ff49014139b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:29.970179 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa9376-667d-45cc-bd1e-ff49014139b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:29.970491 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa9376-667d-45cc-bd1e-ff49014139b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:29.970763 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa9376-667d-45cc-bd1e-ff49014139b1 admin admin] PATH_INFO: `/v3/users/a2efd5f6dc104d089802e376e90916d7` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:29.971884 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daa9376-667d-45cc-bd1e-ff49014139b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2efd5f6dc104d089802e376e90916d7)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:29.973808 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daa9376-667d-45cc-bd1e-ff49014139b1 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:30.008367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:30.036059 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-2daa9376-667d-45cc-bd1e-ff49014139b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2efd5f6dc104d089802e376e90916d7'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:30.036957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-2daa9376-667d-45cc-bd1e-ff49014139b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2efd5f6dc104d089802e376e90916d7'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:30.050485 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-2daa9376-667d-45cc-bd1e-ff49014139b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2efd5f6dc104d089802e376e90916d7'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:30.050485 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-2daa9376-667d-45cc-bd1e-ff49014139b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2efd5f6dc104d089802e376e90916d7'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:30.062528 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-2daa9376-667d-45cc-bd1e-ff49014139b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2efd5f6dc104d089802e376e90916d7'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:30.076505 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 886/1773] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:29 2022] DELETE /identity/v3/users/a2efd5f6dc104d089802e376e90916d7 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:30.509541 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb58066-92e6-4f5e-8152-bc0b3ac9168e admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:30.512365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-deb58066-92e6-4f5e-8152-bc0b3ac9168e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:30.570586 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb58066-92e6-4f5e-8152-bc0b3ac9168e admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:30.694699 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de86994-2fe8-4a96-b0af-752cf0b594de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:30.695356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de86994-2fe8-4a96-b0af-752cf0b594de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:30.695637 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de86994-2fe8-4a96-b0af-752cf0b594de admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:30.695916 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de86994-2fe8-4a96-b0af-752cf0b594de admin admin] PATH_INFO: `/v3/projects/f9a21b664d254044890e6f7f92d2593a` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:30.697061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de86994-2fe8-4a96-b0af-752cf0b594de admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9a21b664d254044890e6f7f92d2593a)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:30.699179 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de86994-2fe8-4a96-b0af-752cf0b594de admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:30.767120 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1de86994-2fe8-4a96-b0af-752cf0b594de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9a21b664d254044890e6f7f92d2593a', 'request_id': 'req-1de86994-2fe8-4a96-b0af-752cf0b594de'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:30.773712 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1de86994-2fe8-4a96-b0af-752cf0b594de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9a21b664d254044890e6f7f92d2593a', 'request_id': 'req-1de86994-2fe8-4a96-b0af-752cf0b594de'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:30.782833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1de86994-2fe8-4a96-b0af-752cf0b594de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9a21b664d254044890e6f7f92d2593a', 'request_id': 'req-1de86994-2fe8-4a96-b0af-752cf0b594de'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:30.801613 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1de86994-2fe8-4a96-b0af-752cf0b594de admin admin] The token cache is being invalidate because project f9a21b664d254044890e6f7f92d2593a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:30.802092 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1de86994-2fe8-4a96-b0af-752cf0b594de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:30.803013 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1de86994-2fe8-4a96-b0af-752cf0b594de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:30.805394 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 888/1774] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:30 2022] DELETE /identity/v3/projects/f9a21b664d254044890e6f7f92d2593a => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:30.811306 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecbc7c-7c04-4306-8deb-eb74058a226a None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:30.811597 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecbc7c-7c04-4306-8deb-eb74058a226a None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:30.811889 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecbc7c-7c04-4306-8deb-eb74058a226a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:30.832293 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-0cecbc7c-7c04-4306-8deb-eb74058a226a None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:30.833522 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-0cecbc7c-7c04-4306-8deb-eb74058a226a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:30.895854 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 887/1775] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:30.903691 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07462655-07e4-49f9-bd29-4772ee2b876b None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:30.903978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07462655-07e4-49f9-bd29-4772ee2b876b None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:30.904253 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07462655-07e4-49f9-bd29-4772ee2b876b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:30.924710 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-07462655-07e4-49f9-bd29-4772ee2b876b None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:30.925949 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-07462655-07e4-49f9-bd29-4772ee2b876b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:30.934939 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 889/1776] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:30.939154 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cecbc7c-7c04-4306-8deb-eb74058a226a None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:30.949669 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cecbc7c-7c04-4306-8deb-eb74058a226a None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:30.954412 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece6a96b-da91-43f6-86d7-c259ebf06153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:30.955024 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6a96b-da91-43f6-86d7-c259ebf06153 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:30.955312 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6a96b-da91-43f6-86d7-c259ebf06153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:30.955586 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6a96b-da91-43f6-86d7-c259ebf06153 admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:30.956241 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece6a96b-da91-43f6-86d7-c259ebf06153 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:34:30.956548 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece6a96b-da91-43f6-86d7-c259ebf06153 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:30.958183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece6a96b-da91-43f6-86d7-c259ebf06153 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:30.970825 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 888/1777] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:34:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:34:30.975192 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07462655-07e4-49f9-bd29-4772ee2b876b None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:30.984493 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07462655-07e4-49f9-bd29-4772ee2b876b None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:30.989049 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417deb22-fead-4dfb-8f09-e7556bf32cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:30.989673 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417deb22-fead-4dfb-8f09-e7556bf32cd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:30.989937 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417deb22-fead-4dfb-8f09-e7556bf32cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:30.990205 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417deb22-fead-4dfb-8f09-e7556bf32cd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:30.990859 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417deb22-fead-4dfb-8f09-e7556bf32cd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:30.992732 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417deb22-fead-4dfb-8f09-e7556bf32cd4 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:31.004718 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:31.023022 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 890/1778] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:30 2022] POST /identity/v3/projects => generated 407 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:31.027184 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece6a96b-da91-43f6-86d7-c259ebf06153 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:31.037606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece6a96b-da91-43f6-86d7-c259ebf06153 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:31.092707 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9cbb05-3f8b-4415-869a-68d3f6671166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:31.093320 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cbb05-3f8b-4415-869a-68d3f6671166 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:31.093604 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cbb05-3f8b-4415-869a-68d3f6671166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:31.093880 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cbb05-3f8b-4415-869a-68d3f6671166 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:31.094551 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9cbb05-3f8b-4415-869a-68d3f6671166 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:31.096481 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9cbb05-3f8b-4415-869a-68d3f6671166 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:31.116585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 889/1779] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:31 2022] POST /identity/v3/users => generated 304 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:31.121244 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417deb22-fead-4dfb-8f09-e7556bf32cd4 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:31.131363 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417deb22-fead-4dfb-8f09-e7556bf32cd4 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:31.136519 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744e78d-2c51-4a1d-af43-f3213cebbcbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:31.137115 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744e78d-2c51-4a1d-af43-f3213cebbcbf admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:31.137394 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744e78d-2c51-4a1d-af43-f3213cebbcbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:31.137657 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744e78d-2c51-4a1d-af43-f3213cebbcbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:31.138320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744e78d-2c51-4a1d-af43-f3213cebbcbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:31.139923 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744e78d-2c51-4a1d-af43-f3213cebbcbf admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:31.149756 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 891/1780] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:31.153967 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9cbb05-3f8b-4415-869a-68d3f6671166 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:31.164747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9cbb05-3f8b-4415-869a-68d3f6671166 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:31.169941 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4255bb8f-0c96-4d97-a7dd-8d721946dfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:31.170591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4255bb8f-0c96-4d97-a7dd-8d721946dfce admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:31.170881 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4255bb8f-0c96-4d97-a7dd-8d721946dfce admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:31.171157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4255bb8f-0c96-4d97-a7dd-8d721946dfce admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:31.171809 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4255bb8f-0c96-4d97-a7dd-8d721946dfce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:31.173409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4255bb8f-0c96-4d97-a7dd-8d721946dfce admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:31.184829 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 890/1781] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:31.189137 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744e78d-2c51-4a1d-af43-f3213cebbcbf admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:31.199527 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744e78d-2c51-4a1d-af43-f3213cebbcbf admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:31.204185 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba994fc1-5d21-4628-bd11-b96c195d1b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:31.204881 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba994fc1-5d21-4628-bd11-b96c195d1b47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:31.205169 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba994fc1-5d21-4628-bd11-b96c195d1b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:31.205446 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba994fc1-5d21-4628-bd11-b96c195d1b47 admin admin] PATH_INFO: `/v3/projects/ff1e8fa9a307434898ca95efa343e07e/users/2f424e051f8f458b937f9bf339fa4e9d/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:31.220643 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba994fc1-5d21-4628-bd11-b96c195d1b47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff1e8fa9a307434898ca95efa343e07e, user_id=2f424e051f8f458b937f9bf339fa4e9d, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:31.223387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba994fc1-5d21-4628-bd11-b96c195d1b47 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:31.237265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 892/1782] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:31 2022] PUT /identity/v3/projects/ff1e8fa9a307434898ca95efa343e07e/users/2f424e051f8f458b937f9bf339fa4e9d/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:31.248183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e2a9bc-1129-4fd4-868f-dbfa78e9aae4 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:31.248503 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e2a9bc-1129-4fd4-868f-dbfa78e9aae4 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:31.248783 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e2a9bc-1129-4fd4-868f-dbfa78e9aae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:31.270000 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:31.279166 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-14e2a9bc-1129-4fd4-868f-dbfa78e9aae4 None None] MFA Rules not processed for user `2f424e051f8f458b937f9bf339fa4e9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:31.281388 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-14e2a9bc-1129-4fd4-868f-dbfa78e9aae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:31.346925 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 891/1783] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:31 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:31.351484 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba994fc1-5d21-4628-bd11-b96c195d1b47 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:31.417636 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba994fc1-5d21-4628-bd11-b96c195d1b47 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:31.422230 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99afb6c4-4e82-4989-b2ea-57023f034434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:31.422876 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afb6c4-4e82-4989-b2ea-57023f034434 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:31.423157 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afb6c4-4e82-4989-b2ea-57023f034434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:31.423426 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afb6c4-4e82-4989-b2ea-57023f034434 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:31.424072 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99afb6c4-4e82-4989-b2ea-57023f034434 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:31.425960 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99afb6c4-4e82-4989-b2ea-57023f034434 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:31.457659 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 893/1784] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:31 2022] POST /identity/v3/projects => generated 407 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:34:31.458910 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e2a9bc-1129-4fd4-868f-dbfa78e9aae4 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:31.472396 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e2a9bc-1129-4fd4-868f-dbfa78e9aae4 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:31.531979 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eadb217-db83-4e9e-bde0-40221e321ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:31.532636 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eadb217-db83-4e9e-bde0-40221e321ad3 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:31.532924 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eadb217-db83-4e9e-bde0-40221e321ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:31.533204 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eadb217-db83-4e9e-bde0-40221e321ad3 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:31.533850 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eadb217-db83-4e9e-bde0-40221e321ad3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:31.535848 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eadb217-db83-4e9e-bde0-40221e321ad3 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:31.557647 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 892/1785] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:31 2022] POST /identity/v3/users => generated 310 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:31.561544 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99afb6c4-4e82-4989-b2ea-57023f034434 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:31.572373 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99afb6c4-4e82-4989-b2ea-57023f034434 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:31.576976 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0447ef-8223-4841-9d21-c528926b7d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:31.577589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0447ef-8223-4841-9d21-c528926b7d2f admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:31.577871 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0447ef-8223-4841-9d21-c528926b7d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:31.578146 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0447ef-8223-4841-9d21-c528926b7d2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:31.578832 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0447ef-8223-4841-9d21-c528926b7d2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:31.580426 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0447ef-8223-4841-9d21-c528926b7d2f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:31.595202 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 894/1786] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:34:31.596306 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eadb217-db83-4e9e-bde0-40221e321ad3 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:31.607355 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eadb217-db83-4e9e-bde0-40221e321ad3 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:31.612005 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78954d3a-85e9-479c-873a-4f3046b95f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:31.612680 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78954d3a-85e9-479c-873a-4f3046b95f04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:31.612961 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78954d3a-85e9-479c-873a-4f3046b95f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:31.613237 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78954d3a-85e9-479c-873a-4f3046b95f04 admin admin] PATH_INFO: `/v3/projects/fd97dba15b5d4d7f870392ca2a33443a/users/647d815092924df6b532ea6fef5408f8/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:31.629086 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78954d3a-85e9-479c-873a-4f3046b95f04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd97dba15b5d4d7f870392ca2a33443a, user_id=647d815092924df6b532ea6fef5408f8, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:31.631798 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78954d3a-85e9-479c-873a-4f3046b95f04 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:31.646126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 893/1787] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:31 2022] PUT /identity/v3/projects/fd97dba15b5d4d7f870392ca2a33443a/users/647d815092924df6b532ea6fef5408f8/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:31.650156 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0447ef-8223-4841-9d21-c528926b7d2f admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:31.663048 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0447ef-8223-4841-9d21-c528926b7d2f admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:31.740902 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4178e276-6b18-4a23-86e6-2c556e7eb61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:31.741640 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178e276-6b18-4a23-86e6-2c556e7eb61d admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:31.742066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178e276-6b18-4a23-86e6-2c556e7eb61d admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:31.742476 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178e276-6b18-4a23-86e6-2c556e7eb61d admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:31.744127 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4178e276-6b18-4a23-86e6-2c556e7eb61d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:31.746030 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4178e276-6b18-4a23-86e6-2c556e7eb61d admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:31.756840 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 895/1788] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:34:31.769428 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78954d3a-85e9-479c-873a-4f3046b95f04 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:31.781278 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78954d3a-85e9-479c-873a-4f3046b95f04 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:31.786228 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a5ae8e-36a5-4a8d-aa8f-494abc5b95f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:31.786917 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5ae8e-36a5-4a8d-aa8f-494abc5b95f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:31.787205 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5ae8e-36a5-4a8d-aa8f-494abc5b95f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:31.787487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5ae8e-36a5-4a8d-aa8f-494abc5b95f8 admin admin] PATH_INFO: `/v3/projects/fd97dba15b5d4d7f870392ca2a33443a/users/647d815092924df6b532ea6fef5408f8/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:31.789391 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a5ae8e-36a5-4a8d-aa8f-494abc5b95f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd97dba15b5d4d7f870392ca2a33443a, user_id=647d815092924df6b532ea6fef5408f8, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:31.792652 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a5ae8e-36a5-4a8d-aa8f-494abc5b95f8 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:31.803212 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 894/1789] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:31 2022] PUT /identity/v3/projects/fd97dba15b5d4d7f870392ca2a33443a/users/647d815092924df6b532ea6fef5408f8/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:31.814659 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6c3ed-9ddc-46af-b606-6693036eb429 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:31.814950 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6c3ed-9ddc-46af-b606-6693036eb429 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:31.815231 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6c3ed-9ddc-46af-b606-6693036eb429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:31.845387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-9bb6c3ed-9ddc-46af-b606-6693036eb429 None None] MFA Rules not processed for user `647d815092924df6b532ea6fef5408f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:31.846668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9bb6c3ed-9ddc-46af-b606-6693036eb429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:31.861675 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:31.922145 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 896/1790] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:31 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:31.934352 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a5ae8e-36a5-4a8d-aa8f-494abc5b95f8 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:31.936098 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-c9a5ae8e-36a5-4a8d-aa8f-494abc5b95f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:31.993954 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a5ae8e-36a5-4a8d-aa8f-494abc5b95f8 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:31.997546 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb00bc7-6346-433e-b8ac-e5ee68e046e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:31.998150 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb00bc7-6346-433e-b8ac-e5ee68e046e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:31.998463 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb00bc7-6346-433e-b8ac-e5ee68e046e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:31.998743 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb00bc7-6346-433e-b8ac-e5ee68e046e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:32.005572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:32.008189 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb00bc7-6346-433e-b8ac-e5ee68e046e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:32.010002 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb00bc7-6346-433e-b8ac-e5ee68e046e7 service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:32.021062 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 895/1791] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:31 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:32.199463 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb6c3ed-9ddc-46af-b606-6693036eb429 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:32.209087 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb6c3ed-9ddc-46af-b606-6693036eb429 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:32.213282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e341a58-cc34-4f0c-8ec1-f382446a4dc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:32.214548 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e341a58-cc34-4f0c-8ec1-f382446a4dc6 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:32.214862 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e341a58-cc34-4f0c-8ec1-f382446a4dc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:32.215136 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e341a58-cc34-4f0c-8ec1-f382446a4dc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:32.271901 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e341a58-cc34-4f0c-8ec1-f382446a4dc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:32.273253 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e341a58-cc34-4f0c-8ec1-f382446a4dc6 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:32.282751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 897/1792] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:32 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:33.402031 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a5ac4-0a72-435e-9b3e-03ab9b8b60d1 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:33.402737 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a5ac4-0a72-435e-9b3e-03ab9b8b60d1 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:33.403220 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a5ac4-0a72-435e-9b3e-03ab9b8b60d1 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:33.405832 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 896/1793] 10.222.0.18 () {54 vars in 921 bytes} [Thu Sep 1 16:34:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:34:33.423684 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ec7a5-caa4-4385-8a86-d4788dfa4b47 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:33.425728 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ec7a5-caa4-4385-8a86-d4788dfa4b47 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:33.426154 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ec7a5-caa4-4385-8a86-d4788dfa4b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:33.513460 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-472ec7a5-caa4-4385-8a86-d4788dfa4b47 None None] MFA Rules not processed for user `278e6d6aed414c758fe3e9f4df6bf81e`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:33.513460 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-472ec7a5-caa4-4385-8a86-d4788dfa4b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:33.663368 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 898/1794] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Sep 1 16:34:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:33.692462 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a5ac4-0a72-435e-9b3e-03ab9b8b60d1 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:33.695335 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-040a5ac4-0a72-435e-9b3e-03ab9b8b60d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:33.732232 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a5ac4-0a72-435e-9b3e-03ab9b8b60d1 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:33.738598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eca3dad-fa0f-4567-8175-8c805c3a93ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:33.739416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca3dad-fa0f-4567-8175-8c805c3a93ca service nova] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:33.739769 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca3dad-fa0f-4567-8175-8c805c3a93ca service nova] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:33.740113 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca3dad-fa0f-4567-8175-8c805c3a93ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:33.751351 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eca3dad-fa0f-4567-8175-8c805c3a93ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:33.753114 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eca3dad-fa0f-4567-8175-8c805c3a93ca service nova] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:33.766160 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 897/1795] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:42.128126 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ec7a5-caa4-4385-8a86-d4788dfa4b47 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:42.186080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:42.218862 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ec7a5-caa4-4385-8a86-d4788dfa4b47 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:42.223845 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd9251-d498-4721-8fdf-d52973cb37c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:42.224501 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd9251-d498-4721-8fdf-d52973cb37c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:42.224810 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd9251-d498-4721-8fdf-d52973cb37c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:42.225088 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd9251-d498-4721-8fdf-d52973cb37c4 admin admin] PATH_INFO: `/v3/users/2f424e051f8f458b937f9bf339fa4e9d` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:42.226307 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cd9251-d498-4721-8fdf-d52973cb37c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f424e051f8f458b937f9bf339fa4e9d)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:42.228543 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cd9251-d498-4721-8fdf-d52973cb37c4 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:42.281366 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f1cd9251-d498-4721-8fdf-d52973cb37c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f424e051f8f458b937f9bf339fa4e9d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:42.292131 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f1cd9251-d498-4721-8fdf-d52973cb37c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f424e051f8f458b937f9bf339fa4e9d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:42.293067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f1cd9251-d498-4721-8fdf-d52973cb37c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f424e051f8f458b937f9bf339fa4e9d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:42.308210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f1cd9251-d498-4721-8fdf-d52973cb37c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f424e051f8f458b937f9bf339fa4e9d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:42.309205 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f1cd9251-d498-4721-8fdf-d52973cb37c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f424e051f8f458b937f9bf339fa4e9d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:42.323632 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 899/1796] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:42 2022] DELETE /identity/v3/users/2f424e051f8f458b937f9bf339fa4e9d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:42.327994 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5e69d-9108-4cb8-8db2-1e594f829a85 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:42.328474 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5e69d-9108-4cb8-8db2-1e594f829a85 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:42.328560 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5e69d-9108-4cb8-8db2-1e594f829a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:42.353060 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-00b5e69d-9108-4cb8-8db2-1e594f829a85 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:42.354523 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-00b5e69d-9108-4cb8-8db2-1e594f829a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:42.417492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 898/1797] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:42.427743 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd9251-d498-4721-8fdf-d52973cb37c4 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:42.429404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f1cd9251-d498-4721-8fdf-d52973cb37c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:42.509247 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd9251-d498-4721-8fdf-d52973cb37c4 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:42.513579 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e0ce1-3452-41f7-b7fc-b0131fc9cdb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:42.514205 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e0ce1-3452-41f7-b7fc-b0131fc9cdb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:42.514533 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e0ce1-3452-41f7-b7fc-b0131fc9cdb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:42.514803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e0ce1-3452-41f7-b7fc-b0131fc9cdb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:42.525600 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8e0ce1-3452-41f7-b7fc-b0131fc9cdb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:42.527121 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8e0ce1-3452-41f7-b7fc-b0131fc9cdb4 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:42.539872 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 900/1798] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:42.862378 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b5e69d-9108-4cb8-8db2-1e594f829a85 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:42.864843 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-00b5e69d-9108-4cb8-8db2-1e594f829a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:42.882606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b5e69d-9108-4cb8-8db2-1e594f829a85 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:42.895688 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d432de68-24b6-45f1-9423-3ef36e7fda13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:42.897234 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432de68-24b6-45f1-9423-3ef36e7fda13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:42.897690 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432de68-24b6-45f1-9423-3ef36e7fda13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:42.898341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432de68-24b6-45f1-9423-3ef36e7fda13 admin admin] PATH_INFO: `/v3/projects/ff1e8fa9a307434898ca95efa343e07e` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:42.902053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d432de68-24b6-45f1-9423-3ef36e7fda13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff1e8fa9a307434898ca95efa343e07e)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:42.905424 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d432de68-24b6-45f1-9423-3ef36e7fda13 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:42.993595 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-d432de68-24b6-45f1-9423-3ef36e7fda13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff1e8fa9a307434898ca95efa343e07e', 'request_id': 'req-d432de68-24b6-45f1-9423-3ef36e7fda13'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:43.004720 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-d432de68-24b6-45f1-9423-3ef36e7fda13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff1e8fa9a307434898ca95efa343e07e', 'request_id': 'req-d432de68-24b6-45f1-9423-3ef36e7fda13'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:43.014910 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-d432de68-24b6-45f1-9423-3ef36e7fda13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff1e8fa9a307434898ca95efa343e07e', 'request_id': 'req-d432de68-24b6-45f1-9423-3ef36e7fda13'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:43.043542 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-d432de68-24b6-45f1-9423-3ef36e7fda13 admin admin] The token cache is being invalidate because project ff1e8fa9a307434898ca95efa343e07e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:43.044265 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-d432de68-24b6-45f1-9423-3ef36e7fda13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:43.045642 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-d432de68-24b6-45f1-9423-3ef36e7fda13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:43.048072 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 899/1799] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:42 2022] DELETE /identity/v3/projects/ff1e8fa9a307434898ca95efa343e07e => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:43.055335 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e0ce1-3452-41f7-b7fc-b0131fc9cdb4 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:43.080880 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e0ce1-3452-41f7-b7fc-b0131fc9cdb4 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:43.177988 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0204553-051d-4948-a530-5ff257ecc1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:43.179852 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0204553-051d-4948-a530-5ff257ecc1bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:43.180565 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0204553-051d-4948-a530-5ff257ecc1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:43.181002 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0204553-051d-4948-a530-5ff257ecc1bf admin admin] PATH_INFO: `/v3/users/647d815092924df6b532ea6fef5408f8` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:43.182770 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0204553-051d-4948-a530-5ff257ecc1bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=647d815092924df6b532ea6fef5408f8)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:43.185376 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0204553-051d-4948-a530-5ff257ecc1bf admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:43.192640 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:43.302937 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e0204553-051d-4948-a530-5ff257ecc1bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '647d815092924df6b532ea6fef5408f8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:43.322455 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e0204553-051d-4948-a530-5ff257ecc1bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '647d815092924df6b532ea6fef5408f8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:43.323301 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e0204553-051d-4948-a530-5ff257ecc1bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '647d815092924df6b532ea6fef5408f8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:43.346532 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e0204553-051d-4948-a530-5ff257ecc1bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '647d815092924df6b532ea6fef5408f8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:43.352257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e0204553-051d-4948-a530-5ff257ecc1bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '647d815092924df6b532ea6fef5408f8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:43.383955 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 901/1800] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:43 2022] DELETE /identity/v3/users/647d815092924df6b532ea6fef5408f8 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:43.785967 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d432de68-24b6-45f1-9423-3ef36e7fda13 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:43.787571 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-d432de68-24b6-45f1-9423-3ef36e7fda13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:43.802369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d432de68-24b6-45f1-9423-3ef36e7fda13 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:43.860158 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e179d61-9f24-4341-80ef-2d2d1d5e3756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:43.860794 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e179d61-9f24-4341-80ef-2d2d1d5e3756 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:43.861077 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e179d61-9f24-4341-80ef-2d2d1d5e3756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:43.861373 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e179d61-9f24-4341-80ef-2d2d1d5e3756 admin admin] PATH_INFO: `/v3/projects/fd97dba15b5d4d7f870392ca2a33443a` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:43.862486 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e179d61-9f24-4341-80ef-2d2d1d5e3756 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd97dba15b5d4d7f870392ca2a33443a)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:43.864409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e179d61-9f24-4341-80ef-2d2d1d5e3756 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:43.872085 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:43.910435 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-6e179d61-9f24-4341-80ef-2d2d1d5e3756 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd97dba15b5d4d7f870392ca2a33443a', 'request_id': 'req-6e179d61-9f24-4341-80ef-2d2d1d5e3756'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:43.917871 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-6e179d61-9f24-4341-80ef-2d2d1d5e3756 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd97dba15b5d4d7f870392ca2a33443a', 'request_id': 'req-6e179d61-9f24-4341-80ef-2d2d1d5e3756'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:43.923981 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-6e179d61-9f24-4341-80ef-2d2d1d5e3756 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd97dba15b5d4d7f870392ca2a33443a', 'request_id': 'req-6e179d61-9f24-4341-80ef-2d2d1d5e3756'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:43.934442 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-6e179d61-9f24-4341-80ef-2d2d1d5e3756 admin admin] The token cache is being invalidate because project fd97dba15b5d4d7f870392ca2a33443a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:43.934734 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-6e179d61-9f24-4341-80ef-2d2d1d5e3756 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:43.935363 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-6e179d61-9f24-4341-80ef-2d2d1d5e3756 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:43.936841 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 900/1801] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:43 2022] DELETE /identity/v3/projects/fd97dba15b5d4d7f870392ca2a33443a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:43.941667 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3c600-bc93-4f3b-be4e-550bc16932e7 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:43.941988 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3c600-bc93-4f3b-be4e-550bc16932e7 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:43.942305 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3c600-bc93-4f3b-be4e-550bc16932e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:43.962701 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-98f3c600-bc93-4f3b-be4e-550bc16932e7 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:43.964459 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-98f3c600-bc93-4f3b-be4e-550bc16932e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:44.031320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 902/1802] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:44.038769 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f33436-c3e4-4841-87bf-edecfd096c1c None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:44.039062 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f33436-c3e4-4841-87bf-edecfd096c1c None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:44.039343 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f33436-c3e4-4841-87bf-edecfd096c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:44.058877 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-d0f33436-c3e4-4841-87bf-edecfd096c1c None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:44.060090 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-d0f33436-c3e4-4841-87bf-edecfd096c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:44.068055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 901/1803] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:44.072194 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3c600-bc93-4f3b-be4e-550bc16932e7 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:44.082451 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3c600-bc93-4f3b-be4e-550bc16932e7 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:44.087292 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846146a-a337-421c-8009-65369f3decc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:44.087778 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846146a-a337-421c-8009-65369f3decc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:44.088059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846146a-a337-421c-8009-65369f3decc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:44.088357 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846146a-a337-421c-8009-65369f3decc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:44.089536 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1846146a-a337-421c-8009-65369f3decc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:34:44.089829 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1846146a-a337-421c-8009-65369f3decc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:44.091559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1846146a-a337-421c-8009-65369f3decc5 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:44.107239 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 903/1804] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:34:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:34:44.108350 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f33436-c3e4-4841-87bf-edecfd096c1c None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:44.117583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f33436-c3e4-4841-87bf-edecfd096c1c None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:44.127732 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ce4ea-ffab-4141-9a9a-7c3f50075894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:44.128422 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ce4ea-ffab-4141-9a9a-7c3f50075894 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:44.128770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ce4ea-ffab-4141-9a9a-7c3f50075894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:44.129109 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ce4ea-ffab-4141-9a9a-7c3f50075894 admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:44.129801 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ce4ea-ffab-4141-9a9a-7c3f50075894 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:44.131839 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ce4ea-ffab-4141-9a9a-7c3f50075894 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:44.162015 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 902/1805] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:44 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:44.166297 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846146a-a337-421c-8009-65369f3decc5 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:44.176170 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846146a-a337-421c-8009-65369f3decc5 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:44.234611 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc57d0c-655a-453b-838f-b1ff5eeeab32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:44.235313 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc57d0c-655a-453b-838f-b1ff5eeeab32 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:44.235566 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc57d0c-655a-453b-838f-b1ff5eeeab32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:44.236175 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc57d0c-655a-453b-838f-b1ff5eeeab32 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:44.236833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc57d0c-655a-453b-838f-b1ff5eeeab32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:44.238783 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc57d0c-655a-453b-838f-b1ff5eeeab32 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:44.255347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:44.261677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 904/1806] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:44 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:44.265781 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ce4ea-ffab-4141-9a9a-7c3f50075894 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:44.276651 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ce4ea-ffab-4141-9a9a-7c3f50075894 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:44.281225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69505b14-8e85-4638-a6e8-26d6e489ff15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:44.281823 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69505b14-8e85-4638-a6e8-26d6e489ff15 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:44.282108 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69505b14-8e85-4638-a6e8-26d6e489ff15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:44.282482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69505b14-8e85-4638-a6e8-26d6e489ff15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:44.283132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69505b14-8e85-4638-a6e8-26d6e489ff15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:44.284734 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69505b14-8e85-4638-a6e8-26d6e489ff15 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:44.294739 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 903/1807] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:44.298640 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc57d0c-655a-453b-838f-b1ff5eeeab32 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:44.309236 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc57d0c-655a-453b-838f-b1ff5eeeab32 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:44.313716 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0344ac-7bb1-41e6-9561-22b2a1adacff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:44.314400 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0344ac-7bb1-41e6-9561-22b2a1adacff admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:44.314683 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0344ac-7bb1-41e6-9561-22b2a1adacff admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:44.314952 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0344ac-7bb1-41e6-9561-22b2a1adacff admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:44.315580 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0344ac-7bb1-41e6-9561-22b2a1adacff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:44.317145 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0344ac-7bb1-41e6-9561-22b2a1adacff admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:44.328128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 905/1808] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:44.332162 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69505b14-8e85-4638-a6e8-26d6e489ff15 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:44.342125 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69505b14-8e85-4638-a6e8-26d6e489ff15 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:44.346800 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666fbe6-d748-4f1d-b828-eb35aa4bab37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:44.347453 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666fbe6-d748-4f1d-b828-eb35aa4bab37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:44.347734 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666fbe6-d748-4f1d-b828-eb35aa4bab37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:44.348007 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666fbe6-d748-4f1d-b828-eb35aa4bab37 admin admin] PATH_INFO: `/v3/projects/a94e7423fcee49d2b35595ed3bed0078/users/8332352b4a2e4fdd9cb0690f152b1009/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:44.362754 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666fbe6-d748-4f1d-b828-eb35aa4bab37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a94e7423fcee49d2b35595ed3bed0078, user_id=8332352b4a2e4fdd9cb0690f152b1009, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:44.365424 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666fbe6-d748-4f1d-b828-eb35aa4bab37 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:44.380781 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 904/1809] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:44 2022] PUT /identity/v3/projects/a94e7423fcee49d2b35595ed3bed0078/users/8332352b4a2e4fdd9cb0690f152b1009/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:44.392944 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbf106-9211-432c-bfb2-666ad481c1fe None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:44.393313 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbf106-9211-432c-bfb2-666ad481c1fe None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:44.393647 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbf106-9211-432c-bfb2-666ad481c1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:44.423935 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-36fbf106-9211-432c-bfb2-666ad481c1fe None None] MFA Rules not processed for user `8332352b4a2e4fdd9cb0690f152b1009`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:44.425282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-36fbf106-9211-432c-bfb2-666ad481c1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:44.502302 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 906/1810] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:34:44.507711 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666fbe6-d748-4f1d-b828-eb35aa4bab37 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:44.589300 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666fbe6-d748-4f1d-b828-eb35aa4bab37 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:44.593891 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85113e9-8dd6-485d-a4d4-1c81af89589e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:44.594536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85113e9-8dd6-485d-a4d4-1c81af89589e admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:44.594821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85113e9-8dd6-485d-a4d4-1c81af89589e admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:44.595098 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85113e9-8dd6-485d-a4d4-1c81af89589e admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:44.595733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85113e9-8dd6-485d-a4d4-1c81af89589e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:44.597753 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85113e9-8dd6-485d-a4d4-1c81af89589e admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:44.626802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 905/1811] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:44 2022] POST /identity/v3/projects => generated 377 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:34:44.630822 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fbf106-9211-432c-bfb2-666ad481c1fe None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:44.641481 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fbf106-9211-432c-bfb2-666ad481c1fe None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:44.707742 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c145965-2d70-48b7-94b8-1c8a706d8bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:44.708362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c145965-2d70-48b7-94b8-1c8a706d8bf2 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:44.708644 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c145965-2d70-48b7-94b8-1c8a706d8bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:44.708919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c145965-2d70-48b7-94b8-1c8a706d8bf2 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:44.709569 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c145965-2d70-48b7-94b8-1c8a706d8bf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:44.711814 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c145965-2d70-48b7-94b8-1c8a706d8bf2 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:44.735428 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 907/1812] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:44 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:44.739208 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85113e9-8dd6-485d-a4d4-1c81af89589e admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:44.751462 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85113e9-8dd6-485d-a4d4-1c81af89589e admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:44.756670 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bccab8-80de-4cd2-abc5-898cf5d09b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:44.757504 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bccab8-80de-4cd2-abc5-898cf5d09b5d admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:44.757976 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bccab8-80de-4cd2-abc5-898cf5d09b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:44.758686 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bccab8-80de-4cd2-abc5-898cf5d09b5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:44.759551 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bccab8-80de-4cd2-abc5-898cf5d09b5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:44.761398 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bccab8-80de-4cd2-abc5-898cf5d09b5d admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:44.778161 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 906/1813] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:44.780038 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c145965-2d70-48b7-94b8-1c8a706d8bf2 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:44.795657 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c145965-2d70-48b7-94b8-1c8a706d8bf2 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:44.805498 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92efce-00b6-419f-9472-3f564947474f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:44.806561 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92efce-00b6-419f-9472-3f564947474f admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:44.806925 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92efce-00b6-419f-9472-3f564947474f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:44.807344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92efce-00b6-419f-9472-3f564947474f admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:44.808693 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c92efce-00b6-419f-9472-3f564947474f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:44.810615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c92efce-00b6-419f-9472-3f564947474f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:44.834470 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 908/1814] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:44.834570 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bccab8-80de-4cd2-abc5-898cf5d09b5d admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:44.840600 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bccab8-80de-4cd2-abc5-898cf5d09b5d admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:44.845370 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a08a6f-7b05-4da4-86be-db5add6182f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:44.846027 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a08a6f-7b05-4da4-86be-db5add6182f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:44.846377 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a08a6f-7b05-4da4-86be-db5add6182f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:44.846659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a08a6f-7b05-4da4-86be-db5add6182f2 admin admin] PATH_INFO: `/v3/projects/0d31238f2a484ee3a24c296d9ed8edfd/users/dc9625190cee46cebffa96872aa38a43/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:44.848994 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:44.865267 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a08a6f-7b05-4da4-86be-db5add6182f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d31238f2a484ee3a24c296d9ed8edfd, user_id=dc9625190cee46cebffa96872aa38a43, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:44.868155 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a08a6f-7b05-4da4-86be-db5add6182f2 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:44.882598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 907/1815] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:44 2022] PUT /identity/v3/projects/0d31238f2a484ee3a24c296d9ed8edfd/users/dc9625190cee46cebffa96872aa38a43/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:44.893024 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c077589-1504-48f6-87b5-ba47b6d79c6f None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:44.893316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c077589-1504-48f6-87b5-ba47b6d79c6f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:44.893595 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c077589-1504-48f6-87b5-ba47b6d79c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:44.921401 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-2c077589-1504-48f6-87b5-ba47b6d79c6f None None] MFA Rules not processed for user `dc9625190cee46cebffa96872aa38a43`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:44.922555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2c077589-1504-48f6-87b5-ba47b6d79c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:44.968732 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:44.997912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 909/1816] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:45.004044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a08a6f-7b05-4da4-86be-db5add6182f2 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:45.089915 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a08a6f-7b05-4da4-86be-db5add6182f2 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:45.095123 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694dd4bb-bce8-4e83-864c-c3bd2eb3b2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:45.095917 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694dd4bb-bce8-4e83-864c-c3bd2eb3b2de admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:45.096351 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694dd4bb-bce8-4e83-864c-c3bd2eb3b2de admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:45.096801 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694dd4bb-bce8-4e83-864c-c3bd2eb3b2de admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:45.097572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694dd4bb-bce8-4e83-864c-c3bd2eb3b2de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:45.099678 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694dd4bb-bce8-4e83-864c-c3bd2eb3b2de admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:45.137741 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c077589-1504-48f6-87b5-ba47b6d79c6f None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:45.138760 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 908/1817] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:45 2022] POST /identity/v3/projects => generated 379 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:45.155488 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c077589-1504-48f6-87b5-ba47b6d79c6f None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:45.253962 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345677f-092c-409b-a18b-bb1ac46e63a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:45.254560 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345677f-092c-409b-a18b-bb1ac46e63a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:45.254899 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345677f-092c-409b-a18b-bb1ac46e63a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:45.255274 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345677f-092c-409b-a18b-bb1ac46e63a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:45.256177 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4345677f-092c-409b-a18b-bb1ac46e63a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:45.258661 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4345677f-092c-409b-a18b-bb1ac46e63a2 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:45.298868 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694dd4bb-bce8-4e83-864c-c3bd2eb3b2de admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:45.318421 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 910/1818] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:45 2022] POST /identity/v3/users => generated 296 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:45.322021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694dd4bb-bce8-4e83-864c-c3bd2eb3b2de admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:45.328544 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245179e-919b-46d5-8f32-36b545c4ae7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:45.329295 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245179e-919b-46d5-8f32-36b545c4ae7d admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:45.329727 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245179e-919b-46d5-8f32-36b545c4ae7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:45.330098 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245179e-919b-46d5-8f32-36b545c4ae7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:45.330910 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5245179e-919b-46d5-8f32-36b545c4ae7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:45.332596 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5245179e-919b-46d5-8f32-36b545c4ae7d admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:45.361176 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345677f-092c-409b-a18b-bb1ac46e63a2 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:45.375201 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 909/1819] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:45.379377 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345677f-092c-409b-a18b-bb1ac46e63a2 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:45.384690 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00eaf5-bda2-4f49-8c88-e65bfecb036e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:45.386514 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00eaf5-bda2-4f49-8c88-e65bfecb036e admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:45.386514 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00eaf5-bda2-4f49-8c88-e65bfecb036e admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:45.386514 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00eaf5-bda2-4f49-8c88-e65bfecb036e admin admin] PATH_INFO: `/v3/projects/904e6fe7d6bc44a593ed617679ce8b00/users/87b913d951ff4dc58e319d78ae5a3b3b/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:45.404337 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c00eaf5-bda2-4f49-8c88-e65bfecb036e admin admin] RBAC: Authorizing `identity:create_grant(project_id=904e6fe7d6bc44a593ed617679ce8b00, user_id=87b913d951ff4dc58e319d78ae5a3b3b, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:45.407879 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c00eaf5-bda2-4f49-8c88-e65bfecb036e admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:45.423550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 911/1820] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:45 2022] PUT /identity/v3/projects/904e6fe7d6bc44a593ed617679ce8b00/users/87b913d951ff4dc58e319d78ae5a3b3b/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:45.427496 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245179e-919b-46d5-8f32-36b545c4ae7d admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:45.438788 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245179e-919b-46d5-8f32-36b545c4ae7d admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:45.518261 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5796513-1434-46cf-9242-8e2946349780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:45.519165 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5796513-1434-46cf-9242-8e2946349780 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:45.519453 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5796513-1434-46cf-9242-8e2946349780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:45.519825 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5796513-1434-46cf-9242-8e2946349780 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:45.521298 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5796513-1434-46cf-9242-8e2946349780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:45.529123 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5796513-1434-46cf-9242-8e2946349780 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:45.549762 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 910/1821] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:34:45.556342 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00eaf5-bda2-4f49-8c88-e65bfecb036e admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:45.576646 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00eaf5-bda2-4f49-8c88-e65bfecb036e admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:45.583509 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa1fd6-e9e5-480d-b09c-7fa1d8e12dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:45.584428 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa1fd6-e9e5-480d-b09c-7fa1d8e12dae admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:45.584717 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa1fd6-e9e5-480d-b09c-7fa1d8e12dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:45.584995 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa1fd6-e9e5-480d-b09c-7fa1d8e12dae admin admin] PATH_INFO: `/v3/projects/904e6fe7d6bc44a593ed617679ce8b00/users/87b913d951ff4dc58e319d78ae5a3b3b/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:45.587217 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa1fd6-e9e5-480d-b09c-7fa1d8e12dae admin admin] RBAC: Authorizing `identity:create_grant(project_id=904e6fe7d6bc44a593ed617679ce8b00, user_id=87b913d951ff4dc58e319d78ae5a3b3b, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:45.590932 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa1fd6-e9e5-480d-b09c-7fa1d8e12dae admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:45.609459 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 912/1822] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:45 2022] PUT /identity/v3/projects/904e6fe7d6bc44a593ed617679ce8b00/users/87b913d951ff4dc58e319d78ae5a3b3b/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:45.626452 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e717d-e61d-450e-8d38-72fca31c215b None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:45.626452 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e717d-e61d-450e-8d38-72fca31c215b None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:45.626452 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e717d-e61d-450e-8d38-72fca31c215b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:45.680863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-b48e717d-e61d-450e-8d38-72fca31c215b None None] MFA Rules not processed for user `87b913d951ff4dc58e319d78ae5a3b3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:45.682272 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-b48e717d-e61d-450e-8d38-72fca31c215b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:45.822913 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:45.877845 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 911/1823] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:45 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:45.966508 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa1fd6-e9e5-480d-b09c-7fa1d8e12dae admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:45.968621 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-35aa1fd6-e9e5-480d-b09c-7fa1d8e12dae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:46.083357 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa1fd6-e9e5-480d-b09c-7fa1d8e12dae admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:46.087493 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5630a9f-04e2-4631-b8a8-dcf03fce60a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:46.088137 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5630a9f-04e2-4631-b8a8-dcf03fce60a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:46.088441 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5630a9f-04e2-4631-b8a8-dcf03fce60a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:46.088720 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5630a9f-04e2-4631-b8a8-dcf03fce60a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:46.099824 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5630a9f-04e2-4631-b8a8-dcf03fce60a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:46.101304 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5630a9f-04e2-4631-b8a8-dcf03fce60a5 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:46.113879 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 913/1824] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:45 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:46.229018 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e717d-e61d-450e-8d38-72fca31c215b None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:46.230905 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-b48e717d-e61d-450e-8d38-72fca31c215b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:46.245996 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e717d-e61d-450e-8d38-72fca31c215b None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:46.250158 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bedf63-3617-43b0-a2cb-734e344189bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:46.251029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bedf63-3617-43b0-a2cb-734e344189bd service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:46.251391 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bedf63-3617-43b0-a2cb-734e344189bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:46.252214 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bedf63-3617-43b0-a2cb-734e344189bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:46.320410 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bedf63-3617-43b0-a2cb-734e344189bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:46.322044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bedf63-3617-43b0-a2cb-734e344189bd service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:46.333682 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 912/1825] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:47.255900 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01916960-c170-4af0-addb-3d02c4765562 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:47.256420 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01916960-c170-4af0-addb-3d02c4765562 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:47.257387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01916960-c170-4af0-addb-3d02c4765562 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:47.259480 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 914/1826] 10.222.0.18 () {54 vars in 921 bytes} [Thu Sep 1 16:34:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:34:47.269896 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5700b8-9ec8-46e9-86b1-3acc93ed82da None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:47.270489 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5700b8-9ec8-46e9-86b1-3acc93ed82da None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:47.270791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5700b8-9ec8-46e9-86b1-3acc93ed82da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:47.300736 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-7d5700b8-9ec8-46e9-86b1-3acc93ed82da None None] MFA Rules not processed for user `278e6d6aed414c758fe3e9f4df6bf81e`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:47.302579 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-7d5700b8-9ec8-46e9-86b1-3acc93ed82da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:47.413901 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 913/1827] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Sep 1 16:34:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:47.437702 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01916960-c170-4af0-addb-3d02c4765562 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:47.439686 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-01916960-c170-4af0-addb-3d02c4765562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:47.461602 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01916960-c170-4af0-addb-3d02c4765562 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:47.480377 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b5b83-f457-4aeb-9208-2f43c0654b66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:47.480999 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b5b83-f457-4aeb-9208-2f43c0654b66 service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:47.481293 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b5b83-f457-4aeb-9208-2f43c0654b66 service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:47.481565 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b5b83-f457-4aeb-9208-2f43c0654b66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:47.491258 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874b5b83-f457-4aeb-9208-2f43c0654b66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:47.492867 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874b5b83-f457-4aeb-9208-2f43c0654b66 service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:47.505128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 915/1828] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:48.633130 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5700b8-9ec8-46e9-86b1-3acc93ed82da None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:48.636723 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-7d5700b8-9ec8-46e9-86b1-3acc93ed82da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:48.756042 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5700b8-9ec8-46e9-86b1-3acc93ed82da None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:48.761521 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc73d7b2-24a7-4fc4-b71c-f1f3e3f31d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:48.762167 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73d7b2-24a7-4fc4-b71c-f1f3e3f31d45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:48.762539 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73d7b2-24a7-4fc4-b71c-f1f3e3f31d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:48.762871 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73d7b2-24a7-4fc4-b71c-f1f3e3f31d45 admin admin] PATH_INFO: `/v3/users/e8f3a2285b9d481990a230b7c389ce70` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:48.764039 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc73d7b2-24a7-4fc4-b71c-f1f3e3f31d45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8f3a2285b9d481990a230b7c389ce70)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:48.766045 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc73d7b2-24a7-4fc4-b71c-f1f3e3f31d45 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:48.778172 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:48.813688 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-fc73d7b2-24a7-4fc4-b71c-f1f3e3f31d45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8f3a2285b9d481990a230b7c389ce70'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:48.814556 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-fc73d7b2-24a7-4fc4-b71c-f1f3e3f31d45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8f3a2285b9d481990a230b7c389ce70'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:48.828022 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-fc73d7b2-24a7-4fc4-b71c-f1f3e3f31d45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8f3a2285b9d481990a230b7c389ce70'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:48.828729 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-fc73d7b2-24a7-4fc4-b71c-f1f3e3f31d45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8f3a2285b9d481990a230b7c389ce70'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:48.841314 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-fc73d7b2-24a7-4fc4-b71c-f1f3e3f31d45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8f3a2285b9d481990a230b7c389ce70'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:48.853588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 914/1829] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:48 2022] DELETE /identity/v3/users/e8f3a2285b9d481990a230b7c389ce70 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:48.858510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36fb70-d48b-4780-b744-63a505ff1dc9 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:48.858970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36fb70-d48b-4780-b744-63a505ff1dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:48.859056 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36fb70-d48b-4780-b744-63a505ff1dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:48.877995 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-5a36fb70-d48b-4780-b744-63a505ff1dc9 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:48.879223 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-5a36fb70-d48b-4780-b744-63a505ff1dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:48.903606 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:48.940439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 916/1830] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:48.954874 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc73d7b2-24a7-4fc4-b71c-f1f3e3f31d45 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:48.956271 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-fc73d7b2-24a7-4fc4-b71c-f1f3e3f31d45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:49.019044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc73d7b2-24a7-4fc4-b71c-f1f3e3f31d45 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:49.022843 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d2250-9381-4ea6-9b41-c192ba8d8868 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:49.023538 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d2250-9381-4ea6-9b41-c192ba8d8868 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:49.023832 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d2250-9381-4ea6-9b41-c192ba8d8868 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:49.024582 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d2250-9381-4ea6-9b41-c192ba8d8868 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:49.034259 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431d2250-9381-4ea6-9b41-c192ba8d8868 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:49.035760 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431d2250-9381-4ea6-9b41-c192ba8d8868 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:49.047194 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 915/1831] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:49.480211 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36fb70-d48b-4780-b744-63a505ff1dc9 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:49.485066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-5a36fb70-d48b-4780-b744-63a505ff1dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:49.500986 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36fb70-d48b-4780-b744-63a505ff1dc9 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:49.508083 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66843f8c-b722-4eb2-8c22-22dff067fc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:49.509089 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66843f8c-b722-4eb2-8c22-22dff067fc9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:49.509740 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66843f8c-b722-4eb2-8c22-22dff067fc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:49.510253 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66843f8c-b722-4eb2-8c22-22dff067fc9a admin admin] PATH_INFO: `/v3/projects/21a5fde5baa94acdace581161f41f356` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:49.531101 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66843f8c-b722-4eb2-8c22-22dff067fc9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=21a5fde5baa94acdace581161f41f356)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:49.533074 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66843f8c-b722-4eb2-8c22-22dff067fc9a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:49.630670 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-66843f8c-b722-4eb2-8c22-22dff067fc9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21a5fde5baa94acdace581161f41f356', 'request_id': 'req-66843f8c-b722-4eb2-8c22-22dff067fc9a'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:49.638677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-66843f8c-b722-4eb2-8c22-22dff067fc9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21a5fde5baa94acdace581161f41f356', 'request_id': 'req-66843f8c-b722-4eb2-8c22-22dff067fc9a'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:49.648046 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-66843f8c-b722-4eb2-8c22-22dff067fc9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21a5fde5baa94acdace581161f41f356', 'request_id': 'req-66843f8c-b722-4eb2-8c22-22dff067fc9a'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:49.654295 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d2250-9381-4ea6-9b41-c192ba8d8868 service neutron] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:49.655954 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-431d2250-9381-4ea6-9b41-c192ba8d8868 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:49.663051 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-66843f8c-b722-4eb2-8c22-22dff067fc9a admin admin] The token cache is being invalidate because project 21a5fde5baa94acdace581161f41f356 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:49.663335 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-66843f8c-b722-4eb2-8c22-22dff067fc9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:49.664160 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-66843f8c-b722-4eb2-8c22-22dff067fc9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:49.665672 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 917/1832] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:49 2022] DELETE /identity/v3/projects/21a5fde5baa94acdace581161f41f356 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:49.670638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66843f8c-b722-4eb2-8c22-22dff067fc9a admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:49.670931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d2250-9381-4ea6-9b41-c192ba8d8868 service neutron] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:49.672239 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-66843f8c-b722-4eb2-8c22-22dff067fc9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:49.756173 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17573dbd-309d-45e4-921c-8cc6d7816212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:49.757037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17573dbd-309d-45e4-921c-8cc6d7816212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:49.757619 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17573dbd-309d-45e4-921c-8cc6d7816212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:49.758041 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17573dbd-309d-45e4-921c-8cc6d7816212 admin admin] PATH_INFO: `/v3/users/8332352b4a2e4fdd9cb0690f152b1009` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:49.759384 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17573dbd-309d-45e4-921c-8cc6d7816212 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8332352b4a2e4fdd9cb0690f152b1009)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:49.761513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17573dbd-309d-45e4-921c-8cc6d7816212 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:49.777262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66843f8c-b722-4eb2-8c22-22dff067fc9a admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:49.787353 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d276c-05df-4e3e-b649-ebf862bb0eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:49.788526 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d276c-05df-4e3e-b649-ebf862bb0eb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:49.789618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d276c-05df-4e3e-b649-ebf862bb0eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:49.790051 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d276c-05df-4e3e-b649-ebf862bb0eb8 admin admin] PATH_INFO: `/v3/users/65114cc4ffba44438bbd2843e86184f8` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:49.792188 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d276c-05df-4e3e-b649-ebf862bb0eb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65114cc4ffba44438bbd2843e86184f8)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:49.795804 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d276c-05df-4e3e-b649-ebf862bb0eb8 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:49.822819 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:49.843225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-17573dbd-309d-45e4-921c-8cc6d7816212 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8332352b4a2e4fdd9cb0690f152b1009'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:49.844425 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-17573dbd-309d-45e4-921c-8cc6d7816212 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8332352b4a2e4fdd9cb0690f152b1009'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:49.853398 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-17573dbd-309d-45e4-921c-8cc6d7816212 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8332352b4a2e4fdd9cb0690f152b1009'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:49.855168 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-17573dbd-309d-45e4-921c-8cc6d7816212 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8332352b4a2e4fdd9cb0690f152b1009'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:49.874619 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-17573dbd-309d-45e4-921c-8cc6d7816212 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8332352b4a2e4fdd9cb0690f152b1009'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:49.887826 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9f5d276c-05df-4e3e-b649-ebf862bb0eb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65114cc4ffba44438bbd2843e86184f8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:49.898355 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 916/1833] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:49 2022] DELETE /identity/v3/users/8332352b4a2e4fdd9cb0690f152b1009 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:49.900043 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9f5d276c-05df-4e3e-b649-ebf862bb0eb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65114cc4ffba44438bbd2843e86184f8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:49.901429 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9f5d276c-05df-4e3e-b649-ebf862bb0eb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65114cc4ffba44438bbd2843e86184f8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:49.903239 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8fe1e-bb2d-4aa4-82e8-e8e92f536270 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:49.903920 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8fe1e-bb2d-4aa4-82e8-e8e92f536270 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:49.904031 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:49.904779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8fe1e-bb2d-4aa4-82e8-e8e92f536270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:49.926516 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9f5d276c-05df-4e3e-b649-ebf862bb0eb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65114cc4ffba44438bbd2843e86184f8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:49.927503 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9f5d276c-05df-4e3e-b649-ebf862bb0eb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65114cc4ffba44438bbd2843e86184f8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:49.934827 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-9ff8fe1e-bb2d-4aa4-82e8-e8e92f536270 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:49.936449 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-9ff8fe1e-bb2d-4aa4-82e8-e8e92f536270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:49.956838 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 918/1834] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:49 2022] DELETE /identity/v3/users/65114cc4ffba44438bbd2843e86184f8 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:50.028240 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 917/1835] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:34:50.043736 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d276c-05df-4e3e-b649-ebf862bb0eb8 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:50.045344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9f5d276c-05df-4e3e-b649-ebf862bb0eb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:50.121768 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d276c-05df-4e3e-b649-ebf862bb0eb8 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:50.125678 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a27a0e4-39e7-46ae-852a-dbd52bbe055c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:50.126321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27a0e4-39e7-46ae-852a-dbd52bbe055c service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:50.126652 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27a0e4-39e7-46ae-852a-dbd52bbe055c service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:50.126959 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27a0e4-39e7-46ae-852a-dbd52bbe055c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:50.222002 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a27a0e4-39e7-46ae-852a-dbd52bbe055c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:50.224381 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a27a0e4-39e7-46ae-852a-dbd52bbe055c service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:50.237002 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 919/1836] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:50.316905 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8fe1e-bb2d-4aa4-82e8-e8e92f536270 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:50.319013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-9ff8fe1e-bb2d-4aa4-82e8-e8e92f536270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:50.332572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8fe1e-bb2d-4aa4-82e8-e8e92f536270 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:50.338471 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757da149-3598-4aeb-aa74-ac0be80a7b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:50.339225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757da149-3598-4aeb-aa74-ac0be80a7b05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:50.339581 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757da149-3598-4aeb-aa74-ac0be80a7b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:50.339948 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757da149-3598-4aeb-aa74-ac0be80a7b05 admin admin] PATH_INFO: `/v3/users/af6d0bcb31a649c790421aa7248ca30e` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:50.341216 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757da149-3598-4aeb-aa74-ac0be80a7b05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af6d0bcb31a649c790421aa7248ca30e)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:50.343658 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757da149-3598-4aeb-aa74-ac0be80a7b05 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:50.415636 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-757da149-3598-4aeb-aa74-ac0be80a7b05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af6d0bcb31a649c790421aa7248ca30e'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:50.416350 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-757da149-3598-4aeb-aa74-ac0be80a7b05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af6d0bcb31a649c790421aa7248ca30e'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:50.427752 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-757da149-3598-4aeb-aa74-ac0be80a7b05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af6d0bcb31a649c790421aa7248ca30e'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:50.428875 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-757da149-3598-4aeb-aa74-ac0be80a7b05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af6d0bcb31a649c790421aa7248ca30e'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:50.444630 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-757da149-3598-4aeb-aa74-ac0be80a7b05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af6d0bcb31a649c790421aa7248ca30e'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:50.460066 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 918/1837] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:50 2022] DELETE /identity/v3/users/af6d0bcb31a649c790421aa7248ca30e => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:50.464526 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35105b4f-91d9-42e5-94f3-7a4ea6eff033 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:50.464823 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35105b4f-91d9-42e5-94f3-7a4ea6eff033 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:50.465138 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35105b4f-91d9-42e5-94f3-7a4ea6eff033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:50.489585 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-35105b4f-91d9-42e5-94f3-7a4ea6eff033 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:50.491595 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-35105b4f-91d9-42e5-94f3-7a4ea6eff033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:50.541897 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757da149-3598-4aeb-aa74-ac0be80a7b05 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:50.543743 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-757da149-3598-4aeb-aa74-ac0be80a7b05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:50.573843 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:50.583407 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 920/1838] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:50.595809 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35105b4f-91d9-42e5-94f3-7a4ea6eff033 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:50.598122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-35105b4f-91d9-42e5-94f3-7a4ea6eff033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:50.634500 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757da149-3598-4aeb-aa74-ac0be80a7b05 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:50.640419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d424d3-338a-4d67-9500-5288afaca16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:50.641134 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d424d3-338a-4d67-9500-5288afaca16a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:50.641504 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:50.641621 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d424d3-338a-4d67-9500-5288afaca16a admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:50.641898 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d424d3-338a-4d67-9500-5288afaca16a admin admin] PATH_INFO: `/v3/projects/a10702e556664b14bd6cb7be929859f6` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:50.643158 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d424d3-338a-4d67-9500-5288afaca16a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a10702e556664b14bd6cb7be929859f6)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:50.645215 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d424d3-338a-4d67-9500-5288afaca16a admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:50.695576 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35105b4f-91d9-42e5-94f3-7a4ea6eff033 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:50.703152 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-13d424d3-338a-4d67-9500-5288afaca16a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a10702e556664b14bd6cb7be929859f6', 'request_id': 'req-13d424d3-338a-4d67-9500-5288afaca16a'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:50.711965 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-13d424d3-338a-4d67-9500-5288afaca16a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a10702e556664b14bd6cb7be929859f6', 'request_id': 'req-13d424d3-338a-4d67-9500-5288afaca16a'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:50.721225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-13d424d3-338a-4d67-9500-5288afaca16a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a10702e556664b14bd6cb7be929859f6', 'request_id': 'req-13d424d3-338a-4d67-9500-5288afaca16a'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:50.733411 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-13d424d3-338a-4d67-9500-5288afaca16a admin admin] The token cache is being invalidate because project a10702e556664b14bd6cb7be929859f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:50.733411 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-13d424d3-338a-4d67-9500-5288afaca16a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:50.734039 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-13d424d3-338a-4d67-9500-5288afaca16a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:50.737670 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 919/1839] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:50 2022] DELETE /identity/v3/projects/a10702e556664b14bd6cb7be929859f6 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:50.742279 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d27ca5-5cdd-44a6-aca3-4777c4bfbc1e None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:50.742994 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d27ca5-5cdd-44a6-aca3-4777c4bfbc1e None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:50.742994 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d27ca5-5cdd-44a6-aca3-4777c4bfbc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:50.763832 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-90d27ca5-5cdd-44a6-aca3-4777c4bfbc1e None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:50.765238 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-90d27ca5-5cdd-44a6-aca3-4777c4bfbc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:50.767906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d7e786-3d64-4cbf-9b01-8ba7ad642bb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:50.768364 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7e786-3d64-4cbf-9b01-8ba7ad642bb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:50.769102 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7e786-3d64-4cbf-9b01-8ba7ad642bb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:50.769486 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7e786-3d64-4cbf-9b01-8ba7ad642bb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:50.771902 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-76d7e786-3d64-4cbf-9b01-8ba7ad642bb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:50.851990 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 920/1840] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:50.853607 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a259a-43f2-4a9e-a72c-ac867449e68a None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:50.855977 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a259a-43f2-4a9e-a72c-ac867449e68a None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:50.856329 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a259a-43f2-4a9e-a72c-ac867449e68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:50.858215 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d7e786-3d64-4cbf-9b01-8ba7ad642bb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:50.859910 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d7e786-3d64-4cbf-9b01-8ba7ad642bb4 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:50.873163 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 921/1841] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:34:50.878568 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-3b0a259a-43f2-4a9e-a72c-ac867449e68a None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:50.881562 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-3b0a259a-43f2-4a9e-a72c-ac867449e68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:50.889001 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 921/1842] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:34:50.893259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d7e786-3d64-4cbf-9b01-8ba7ad642bb4 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:50.904064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d7e786-3d64-4cbf-9b01-8ba7ad642bb4 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:50.909023 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f19b212-054b-43c7-8469-1132b802e254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:50.909799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19b212-054b-43c7-8469-1132b802e254 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:50.910182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19b212-054b-43c7-8469-1132b802e254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:50.910590 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19b212-054b-43c7-8469-1132b802e254 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:50.911304 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f19b212-054b-43c7-8469-1132b802e254 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:34:50.911674 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f19b212-054b-43c7-8469-1132b802e254 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:50.913324 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f19b212-054b-43c7-8469-1132b802e254 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:50.922157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a259a-43f2-4a9e-a72c-ac867449e68a None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:50.924696 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-3b0a259a-43f2-4a9e-a72c-ac867449e68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:50.930124 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 922/1843] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:34:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:34:50.931141 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f19b212-054b-43c7-8469-1132b802e254 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:50.941114 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a259a-43f2-4a9e-a72c-ac867449e68a None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:50.942540 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f19b212-054b-43c7-8469-1132b802e254 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:50.948377 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb5be0-a575-47ba-9f26-690dc2de06ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:50.948982 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb5be0-a575-47ba-9f26-690dc2de06ac admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:50.949329 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb5be0-a575-47ba-9f26-690dc2de06ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:50.949667 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb5be0-a575-47ba-9f26-690dc2de06ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:50.950402 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb5be0-a575-47ba-9f26-690dc2de06ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:50.953007 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb5be0-a575-47ba-9f26-690dc2de06ac admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:50.953812 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3089d-6d90-423c-938c-b43e1f20051f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:50.954582 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3089d-6d90-423c-938c-b43e1f20051f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:50.955577 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3089d-6d90-423c-938c-b43e1f20051f admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:50.956323 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3089d-6d90-423c-938c-b43e1f20051f admin admin] PATH_INFO: `/v3/projects/a94e7423fcee49d2b35595ed3bed0078` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:50.959507 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee3089d-6d90-423c-938c-b43e1f20051f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a94e7423fcee49d2b35595ed3bed0078)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:50.963863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee3089d-6d90-423c-938c-b43e1f20051f admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:50.993437 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 923/1844] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:50 2022] POST /identity/v3/projects => generated 363 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:50.998444 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb5be0-a575-47ba-9f26-690dc2de06ac admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:51.072358 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:51.083924 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-2ee3089d-6d90-423c-938c-b43e1f20051f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a94e7423fcee49d2b35595ed3bed0078', 'request_id': 'req-2ee3089d-6d90-423c-938c-b43e1f20051f'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:51.097595 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-2ee3089d-6d90-423c-938c-b43e1f20051f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a94e7423fcee49d2b35595ed3bed0078', 'request_id': 'req-2ee3089d-6d90-423c-938c-b43e1f20051f'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:51.119183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:51.136820 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb5be0-a575-47ba-9f26-690dc2de06ac admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:51.145004 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-2ee3089d-6d90-423c-938c-b43e1f20051f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a94e7423fcee49d2b35595ed3bed0078', 'request_id': 'req-2ee3089d-6d90-423c-938c-b43e1f20051f'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:51.173088 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-2ee3089d-6d90-423c-938c-b43e1f20051f admin admin] The token cache is being invalidate because project a94e7423fcee49d2b35595ed3bed0078 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:51.173433 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-2ee3089d-6d90-423c-938c-b43e1f20051f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:51.178918 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-2ee3089d-6d90-423c-938c-b43e1f20051f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:51.181548 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 922/1845] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:50 2022] DELETE /identity/v3/projects/a94e7423fcee49d2b35595ed3bed0078 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:51.186343 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3089d-6d90-423c-938c-b43e1f20051f admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:51.187947 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-2ee3089d-6d90-423c-938c-b43e1f20051f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:51.241730 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b87362-cdff-4222-8f9e-4364e7fce0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:51.247432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b87362-cdff-4222-8f9e-4364e7fce0e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:51.247989 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b87362-cdff-4222-8f9e-4364e7fce0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:51.248423 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b87362-cdff-4222-8f9e-4364e7fce0e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:51.250122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b87362-cdff-4222-8f9e-4364e7fce0e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:51.252581 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b87362-cdff-4222-8f9e-4364e7fce0e4 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:51.281534 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3089d-6d90-423c-938c-b43e1f20051f admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:51.301241 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 924/1846] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:50 2022] POST /identity/v3/users => generated 282 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:51.302433 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b87362-cdff-4222-8f9e-4364e7fce0e4 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:51.304885 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-78b87362-cdff-4222-8f9e-4364e7fce0e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:51.322181 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdcfb4-fd91-4dcd-81a6-8348527a3782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:51.322951 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b87362-cdff-4222-8f9e-4364e7fce0e4 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:51.323069 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdcfb4-fd91-4dcd-81a6-8348527a3782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:51.323269 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdcfb4-fd91-4dcd-81a6-8348527a3782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:51.323541 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdcfb4-fd91-4dcd-81a6-8348527a3782 admin admin] PATH_INFO: `/v3/users/dc9625190cee46cebffa96872aa38a43` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:51.327362 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cdcfb4-fd91-4dcd-81a6-8348527a3782 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc9625190cee46cebffa96872aa38a43)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:51.329410 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c06b92-2e11-4384-a125-b344809d19fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:51.330699 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c06b92-2e11-4384-a125-b344809d19fd admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:51.331289 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c06b92-2e11-4384-a125-b344809d19fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:51.331956 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c06b92-2e11-4384-a125-b344809d19fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:51.333437 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c06b92-2e11-4384-a125-b344809d19fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:51.333570 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cdcfb4-fd91-4dcd-81a6-8348527a3782 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:51.336027 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c06b92-2e11-4384-a125-b344809d19fd admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:51.352752 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 925/1847] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:51 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:51.354017 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c06b92-2e11-4384-a125-b344809d19fd admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:51.356489 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f5c06b92-2e11-4384-a125-b344809d19fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:51.372439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c06b92-2e11-4384-a125-b344809d19fd admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:51.378554 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2d374-8654-4e23-9255-99a33296041f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:51.379508 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2d374-8654-4e23-9255-99a33296041f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:51.380013 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2d374-8654-4e23-9255-99a33296041f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:51.380486 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2d374-8654-4e23-9255-99a33296041f admin admin] PATH_INFO: `/v3/projects/0a1b9052f76f45fdb904e92d4bd3949f` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:51.382043 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2d374-8654-4e23-9255-99a33296041f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a1b9052f76f45fdb904e92d4bd3949f)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:51.384349 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2d374-8654-4e23-9255-99a33296041f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:51.457379 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-18cdcfb4-fd91-4dcd-81a6-8348527a3782 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc9625190cee46cebffa96872aa38a43'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:51.458540 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-18cdcfb4-fd91-4dcd-81a6-8348527a3782 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc9625190cee46cebffa96872aa38a43'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:51.466777 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-18cdcfb4-fd91-4dcd-81a6-8348527a3782 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc9625190cee46cebffa96872aa38a43'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:51.466943 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3ef2d374-8654-4e23-9255-99a33296041f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a1b9052f76f45fdb904e92d4bd3949f', 'request_id': 'req-3ef2d374-8654-4e23-9255-99a33296041f'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:51.467598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-18cdcfb4-fd91-4dcd-81a6-8348527a3782 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc9625190cee46cebffa96872aa38a43'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:51.474440 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3ef2d374-8654-4e23-9255-99a33296041f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a1b9052f76f45fdb904e92d4bd3949f', 'request_id': 'req-3ef2d374-8654-4e23-9255-99a33296041f'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:51.483502 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3ef2d374-8654-4e23-9255-99a33296041f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a1b9052f76f45fdb904e92d4bd3949f', 'request_id': 'req-3ef2d374-8654-4e23-9255-99a33296041f'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:51.487042 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-18cdcfb4-fd91-4dcd-81a6-8348527a3782 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc9625190cee46cebffa96872aa38a43'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:51.497968 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3ef2d374-8654-4e23-9255-99a33296041f admin admin] The token cache is being invalidate because project 0a1b9052f76f45fdb904e92d4bd3949f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:51.498534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3ef2d374-8654-4e23-9255-99a33296041f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:51.499398 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3ef2d374-8654-4e23-9255-99a33296041f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:51.504948 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 923/1848] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:51 2022] DELETE /identity/v3/users/dc9625190cee46cebffa96872aa38a43 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:51.504948 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdcfb4-fd91-4dcd-81a6-8348527a3782 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:51.505162 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-18cdcfb4-fd91-4dcd-81a6-8348527a3782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:51.510858 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 926/1849] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:51 2022] DELETE /identity/v3/projects/0a1b9052f76f45fdb904e92d4bd3949f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:51.517160 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2d374-8654-4e23-9255-99a33296041f admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:51.520760 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3ef2d374-8654-4e23-9255-99a33296041f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:51.588721 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:51.614641 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdcfb4-fd91-4dcd-81a6-8348527a3782 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:51.622513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f89da2-47e2-43dc-b46e-e0aa774862aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:51.623140 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f89da2-47e2-43dc-b46e-e0aa774862aa admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:51.623438 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f89da2-47e2-43dc-b46e-e0aa774862aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:51.623699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f89da2-47e2-43dc-b46e-e0aa774862aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:51.624349 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f89da2-47e2-43dc-b46e-e0aa774862aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:51.626088 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f89da2-47e2-43dc-b46e-e0aa774862aa admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:51.636418 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2d374-8654-4e23-9255-99a33296041f admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:51.640461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 924/1850] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:51 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:51.641711 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f89da2-47e2-43dc-b46e-e0aa774862aa admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:51.642203 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8302264-9a06-4f37-ba3d-5573b5358a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:51.643008 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8302264-9a06-4f37-ba3d-5573b5358a5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:51.643295 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8302264-9a06-4f37-ba3d-5573b5358a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:51.643573 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8302264-9a06-4f37-ba3d-5573b5358a5c admin admin] PATH_INFO: `/v3/users/7b5bb04abbbf4d0781ca3a47cd882555` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:51.645882 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-04f89da2-47e2-43dc-b46e-e0aa774862aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:51.647027 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8302264-9a06-4f37-ba3d-5573b5358a5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b5bb04abbbf4d0781ca3a47cd882555)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:51.649203 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8302264-9a06-4f37-ba3d-5573b5358a5c admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:51.652945 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:51.667141 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f89da2-47e2-43dc-b46e-e0aa774862aa admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:51.691891 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cefb4b-a848-4fde-9a6d-3927a3e31125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:51.692592 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cefb4b-a848-4fde-9a6d-3927a3e31125 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:51.692891 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cefb4b-a848-4fde-9a6d-3927a3e31125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:51.693192 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cefb4b-a848-4fde-9a6d-3927a3e31125 admin admin] PATH_INFO: `/v3/projects/4a2d39ce733640059b3d367e21ba9caa/users/3e47f24f10de4f8da90de9cf111106eb/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:51.718726 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cefb4b-a848-4fde-9a6d-3927a3e31125 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a2d39ce733640059b3d367e21ba9caa, user_id=3e47f24f10de4f8da90de9cf111106eb, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:51.722445 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cefb4b-a848-4fde-9a6d-3927a3e31125 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:51.746069 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 925/1851] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:51 2022] PUT /identity/v3/projects/4a2d39ce733640059b3d367e21ba9caa/users/3e47f24f10de4f8da90de9cf111106eb/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:51.756394 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc52840-86ec-43d8-a46b-c110931adecd None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:51.759741 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc52840-86ec-43d8-a46b-c110931adecd None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:51.760528 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc52840-86ec-43d8-a46b-c110931adecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:51.764223 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b8302264-9a06-4f37-ba3d-5573b5358a5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b5bb04abbbf4d0781ca3a47cd882555'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:51.784878 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b8302264-9a06-4f37-ba3d-5573b5358a5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b5bb04abbbf4d0781ca3a47cd882555'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:51.785463 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b8302264-9a06-4f37-ba3d-5573b5358a5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b5bb04abbbf4d0781ca3a47cd882555'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:51.807624 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b8302264-9a06-4f37-ba3d-5573b5358a5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b5bb04abbbf4d0781ca3a47cd882555'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:51.809689 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b8302264-9a06-4f37-ba3d-5573b5358a5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b5bb04abbbf4d0781ca3a47cd882555'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:51.827566 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-8bc52840-86ec-43d8-a46b-c110931adecd None None] MFA Rules not processed for user `3e47f24f10de4f8da90de9cf111106eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:51.829537 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-8bc52840-86ec-43d8-a46b-c110931adecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:51.840317 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 927/1852] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:51 2022] DELETE /identity/v3/users/7b5bb04abbbf4d0781ca3a47cd882555 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:51.899330 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8302264-9a06-4f37-ba3d-5573b5358a5c admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:51.903080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b8302264-9a06-4f37-ba3d-5573b5358a5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:52.012682 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 926/1853] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:34:52.019999 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc52840-86ec-43d8-a46b-c110931adecd None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:52.042965 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-8bc52840-86ec-43d8-a46b-c110931adecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:52.054221 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8302264-9a06-4f37-ba3d-5573b5358a5c admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:52.060254 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc7788f-99e2-4a6e-9350-d356f670969d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:52.061220 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7788f-99e2-4a6e-9350-d356f670969d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:52.061642 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7788f-99e2-4a6e-9350-d356f670969d admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:52.062121 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7788f-99e2-4a6e-9350-d356f670969d admin admin] PATH_INFO: `/v3/projects/0d31238f2a484ee3a24c296d9ed8edfd` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:52.063422 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc7788f-99e2-4a6e-9350-d356f670969d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d31238f2a484ee3a24c296d9ed8edfd)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:52.065485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc7788f-99e2-4a6e-9350-d356f670969d admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:52.159951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc52840-86ec-43d8-a46b-c110931adecd None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:52.190559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-dbc7788f-99e2-4a6e-9350-d356f670969d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d31238f2a484ee3a24c296d9ed8edfd', 'request_id': 'req-dbc7788f-99e2-4a6e-9350-d356f670969d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:52.237325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-dbc7788f-99e2-4a6e-9350-d356f670969d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d31238f2a484ee3a24c296d9ed8edfd', 'request_id': 'req-dbc7788f-99e2-4a6e-9350-d356f670969d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:52.274887 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-dbc7788f-99e2-4a6e-9350-d356f670969d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d31238f2a484ee3a24c296d9ed8edfd', 'request_id': 'req-dbc7788f-99e2-4a6e-9350-d356f670969d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:52.310385 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:52.320817 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-dbc7788f-99e2-4a6e-9350-d356f670969d admin admin] The token cache is being invalidate because project 0d31238f2a484ee3a24c296d9ed8edfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:52.321172 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-dbc7788f-99e2-4a6e-9350-d356f670969d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:52.323649 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-dbc7788f-99e2-4a6e-9350-d356f670969d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:52.326350 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 928/1854] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:51 2022] DELETE /identity/v3/projects/0d31238f2a484ee3a24c296d9ed8edfd => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:52.330049 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc7788f-99e2-4a6e-9350-d356f670969d admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:52.330728 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3d60c-3cfd-4f5f-a7a9-6541d12eb449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:52.333314 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3d60c-3cfd-4f5f-a7a9-6541d12eb449 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:52.333609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3d60c-3cfd-4f5f-a7a9-6541d12eb449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:52.333899 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3d60c-3cfd-4f5f-a7a9-6541d12eb449 admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:52.335119 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-dbc7788f-99e2-4a6e-9350-d356f670969d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:52.336677 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c3d60c-3cfd-4f5f-a7a9-6541d12eb449 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:52.343611 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c3d60c-3cfd-4f5f-a7a9-6541d12eb449 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:52.367092 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc7788f-99e2-4a6e-9350-d356f670969d admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:52.381999 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e7ebe-27fa-4aa3-8553-ecb9822f60e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:52.383242 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e7ebe-27fa-4aa3-8553-ecb9822f60e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:52.383469 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e7ebe-27fa-4aa3-8553-ecb9822f60e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:52.383750 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e7ebe-27fa-4aa3-8553-ecb9822f60e3 admin admin] PATH_INFO: `/v3/users/87b913d951ff4dc58e319d78ae5a3b3b` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:52.387347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e7ebe-27fa-4aa3-8553-ecb9822f60e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87b913d951ff4dc58e319d78ae5a3b3b)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:52.389339 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e7ebe-27fa-4aa3-8553-ecb9822f60e3 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:52.396843 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:52.410498 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 927/1855] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:52 2022] POST /identity/v3/projects => generated 363 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:52.414692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3d60c-3cfd-4f5f-a7a9-6541d12eb449 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:52.417347 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-11c3d60c-3cfd-4f5f-a7a9-6541d12eb449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:52.490349 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-a07e7ebe-27fa-4aa3-8553-ecb9822f60e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87b913d951ff4dc58e319d78ae5a3b3b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:52.508181 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-a07e7ebe-27fa-4aa3-8553-ecb9822f60e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87b913d951ff4dc58e319d78ae5a3b3b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:52.508853 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-a07e7ebe-27fa-4aa3-8553-ecb9822f60e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87b913d951ff4dc58e319d78ae5a3b3b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:52.531365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-a07e7ebe-27fa-4aa3-8553-ecb9822f60e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87b913d951ff4dc58e319d78ae5a3b3b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:52.533519 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-a07e7ebe-27fa-4aa3-8553-ecb9822f60e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87b913d951ff4dc58e319d78ae5a3b3b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:52.567738 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 929/1856] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:52 2022] DELETE /identity/v3/users/87b913d951ff4dc58e319d78ae5a3b3b => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:52.599761 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3d60c-3cfd-4f5f-a7a9-6541d12eb449 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:52.748860 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903490d9-300b-4e65-8cdd-532295dc6ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:52.749580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903490d9-300b-4e65-8cdd-532295dc6ef7 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:52.749836 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903490d9-300b-4e65-8cdd-532295dc6ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:52.750152 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903490d9-300b-4e65-8cdd-532295dc6ef7 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:52.750745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903490d9-300b-4e65-8cdd-532295dc6ef7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:52.758372 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903490d9-300b-4e65-8cdd-532295dc6ef7 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:52.808124 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 928/1857] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:52 2022] POST /identity/v3/users => generated 288 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:52.809457 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e7ebe-27fa-4aa3-8553-ecb9822f60e3 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:52.831134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a07e7ebe-27fa-4aa3-8553-ecb9822f60e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:52.853661 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e7ebe-27fa-4aa3-8553-ecb9822f60e3 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:52.871366 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386435f-43dc-4465-9964-bae2bed72b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:52.872005 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386435f-43dc-4465-9964-bae2bed72b88 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:52.872299 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386435f-43dc-4465-9964-bae2bed72b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:52.872580 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386435f-43dc-4465-9964-bae2bed72b88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:52.873238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b386435f-43dc-4465-9964-bae2bed72b88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:52.874898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b386435f-43dc-4465-9964-bae2bed72b88 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:52.899778 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 930/1858] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:52.904150 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903490d9-300b-4e65-8cdd-532295dc6ef7 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:52.939278 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903490d9-300b-4e65-8cdd-532295dc6ef7 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:52.952699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30007a89-3cb0-4b2e-a2cf-fdf1fcf5cc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:52.954771 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30007a89-3cb0-4b2e-a2cf-fdf1fcf5cc8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:52.955549 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30007a89-3cb0-4b2e-a2cf-fdf1fcf5cc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:52.956340 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30007a89-3cb0-4b2e-a2cf-fdf1fcf5cc8f admin admin] PATH_INFO: `/v3/projects/892540cb67fb41cda8dbb348a9ed5e8d/users/2142cb3fe0b9436997488d840c46ccad/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:53.012172 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30007a89-3cb0-4b2e-a2cf-fdf1fcf5cc8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=892540cb67fb41cda8dbb348a9ed5e8d, user_id=2142cb3fe0b9436997488d840c46ccad, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:53.016765 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30007a89-3cb0-4b2e-a2cf-fdf1fcf5cc8f admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:53.047146 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 929/1859] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:52 2022] PUT /identity/v3/projects/892540cb67fb41cda8dbb348a9ed5e8d/users/2142cb3fe0b9436997488d840c46ccad/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:53.049193 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386435f-43dc-4465-9964-bae2bed72b88 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:53.085517 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386435f-43dc-4465-9964-bae2bed72b88 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:53.165360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c786aa9-9830-4292-b82a-3b58f65f7237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:53.166001 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c786aa9-9830-4292-b82a-3b58f65f7237 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:53.166469 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c786aa9-9830-4292-b82a-3b58f65f7237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:53.167075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c786aa9-9830-4292-b82a-3b58f65f7237 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:53.167817 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c786aa9-9830-4292-b82a-3b58f65f7237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:53.169953 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c786aa9-9830-4292-b82a-3b58f65f7237 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:53.184648 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 931/1860] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:53 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:53.186336 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30007a89-3cb0-4b2e-a2cf-fdf1fcf5cc8f admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:53.200586 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30007a89-3cb0-4b2e-a2cf-fdf1fcf5cc8f admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:53.207017 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad7b41-38aa-43c4-8082-6843732ff29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:53.208956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad7b41-38aa-43c4-8082-6843732ff29c admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:53.208956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad7b41-38aa-43c4-8082-6843732ff29c admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:53.208956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad7b41-38aa-43c4-8082-6843732ff29c admin admin] PATH_INFO: `/v3/projects/892540cb67fb41cda8dbb348a9ed5e8d/users/2142cb3fe0b9436997488d840c46ccad/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:53.223278 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ad7b41-38aa-43c4-8082-6843732ff29c admin admin] RBAC: Authorizing `identity:create_grant(project_id=892540cb67fb41cda8dbb348a9ed5e8d, user_id=2142cb3fe0b9436997488d840c46ccad, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:53.240187 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ad7b41-38aa-43c4-8082-6843732ff29c admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:53.258414 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 930/1861] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:53 2022] PUT /identity/v3/projects/892540cb67fb41cda8dbb348a9ed5e8d/users/2142cb3fe0b9436997488d840c46ccad/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:53.270637 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84544c56-a946-4f1f-95a8-49f443e231e5 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:53.271057 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84544c56-a946-4f1f-95a8-49f443e231e5 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:53.271341 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84544c56-a946-4f1f-95a8-49f443e231e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:53.301651 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-84544c56-a946-4f1f-95a8-49f443e231e5 None None] MFA Rules not processed for user `2142cb3fe0b9436997488d840c46ccad`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:53.302910 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-84544c56-a946-4f1f-95a8-49f443e231e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:53.372729 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:53.383510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad7b41-38aa-43c4-8082-6843732ff29c admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:53.386717 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-97ad7b41-38aa-43c4-8082-6843732ff29c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:53.394888 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 932/1862] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:53 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:53.394888 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84544c56-a946-4f1f-95a8-49f443e231e5 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:53.394888 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-84544c56-a946-4f1f-95a8-49f443e231e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:53.452048 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:53.466800 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84544c56-a946-4f1f-95a8-49f443e231e5 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:53.471730 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586610cc-a854-4edb-9be6-b55a683b8f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:53.472120 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad7b41-38aa-43c4-8082-6843732ff29c admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:53.472415 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586610cc-a854-4edb-9be6-b55a683b8f0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:53.472767 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586610cc-a854-4edb-9be6-b55a683b8f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:53.472972 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586610cc-a854-4edb-9be6-b55a683b8f0a admin admin] PATH_INFO: `/v3/projects/710f523c46724172a8cc4726770c525a` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:53.474081 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586610cc-a854-4edb-9be6-b55a683b8f0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=710f523c46724172a8cc4726770c525a)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:53.476036 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586610cc-a854-4edb-9be6-b55a683b8f0a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:53.476755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca42d56-3a81-43d8-82f6-6fa5a7e7655f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:53.477460 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca42d56-3a81-43d8-82f6-6fa5a7e7655f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:53.477666 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca42d56-3a81-43d8-82f6-6fa5a7e7655f admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:53.477947 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca42d56-3a81-43d8-82f6-6fa5a7e7655f admin admin] PATH_INFO: `/v3/projects/904e6fe7d6bc44a593ed617679ce8b00` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:53.479180 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca42d56-3a81-43d8-82f6-6fa5a7e7655f admin admin] RBAC: Authorizing `identity:delete_project(project_id=904e6fe7d6bc44a593ed617679ce8b00)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:53.481339 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca42d56-3a81-43d8-82f6-6fa5a7e7655f admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:53.525517 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-586610cc-a854-4edb-9be6-b55a683b8f0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '710f523c46724172a8cc4726770c525a', 'request_id': 'req-586610cc-a854-4edb-9be6-b55a683b8f0a'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:53.529356 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5ca42d56-3a81-43d8-82f6-6fa5a7e7655f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '904e6fe7d6bc44a593ed617679ce8b00', 'request_id': 'req-5ca42d56-3a81-43d8-82f6-6fa5a7e7655f'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:53.531889 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-586610cc-a854-4edb-9be6-b55a683b8f0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '710f523c46724172a8cc4726770c525a', 'request_id': 'req-586610cc-a854-4edb-9be6-b55a683b8f0a'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:53.536898 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5ca42d56-3a81-43d8-82f6-6fa5a7e7655f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '904e6fe7d6bc44a593ed617679ce8b00', 'request_id': 'req-5ca42d56-3a81-43d8-82f6-6fa5a7e7655f'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:53.538875 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-586610cc-a854-4edb-9be6-b55a683b8f0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '710f523c46724172a8cc4726770c525a', 'request_id': 'req-586610cc-a854-4edb-9be6-b55a683b8f0a'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:53.543273 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5ca42d56-3a81-43d8-82f6-6fa5a7e7655f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '904e6fe7d6bc44a593ed617679ce8b00', 'request_id': 'req-5ca42d56-3a81-43d8-82f6-6fa5a7e7655f'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:53.549925 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-586610cc-a854-4edb-9be6-b55a683b8f0a admin admin] The token cache is being invalidate because project 710f523c46724172a8cc4726770c525a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:53.550216 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-586610cc-a854-4edb-9be6-b55a683b8f0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:53.550885 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-586610cc-a854-4edb-9be6-b55a683b8f0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:53.552406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 933/1863] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:53 2022] DELETE /identity/v3/projects/710f523c46724172a8cc4726770c525a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:53.553164 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586610cc-a854-4edb-9be6-b55a683b8f0a admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:53.554059 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5ca42d56-3a81-43d8-82f6-6fa5a7e7655f admin admin] The token cache is being invalidate because project 904e6fe7d6bc44a593ed617679ce8b00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:53.554445 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5ca42d56-3a81-43d8-82f6-6fa5a7e7655f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:53.554550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-586610cc-a854-4edb-9be6-b55a683b8f0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:53.555218 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5ca42d56-3a81-43d8-82f6-6fa5a7e7655f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:53.557548 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 931/1864] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:53 2022] DELETE /identity/v3/projects/904e6fe7d6bc44a593ed617679ce8b00 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:53.558873 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedb7ba-66be-4627-86cd-ce5cf2a8d220 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:53.559166 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedb7ba-66be-4627-86cd-ce5cf2a8d220 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:53.559443 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedb7ba-66be-4627-86cd-ce5cf2a8d220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:53.580025 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-afedb7ba-66be-4627-86cd-ce5cf2a8d220 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:53.581232 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-afedb7ba-66be-4627-86cd-ce5cf2a8d220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:53.611658 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586610cc-a854-4edb-9be6-b55a683b8f0a admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:53.641001 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 932/1865] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:53.642359 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef235e-e782-4035-b887-bd12a478c9e9 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:53.642644 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef235e-e782-4035-b887-bd12a478c9e9 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:53.642950 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef235e-e782-4035-b887-bd12a478c9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:53.657417 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:53.664667 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-a7ef235e-e782-4035-b887-bd12a478c9e9 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:53.666067 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-a7ef235e-e782-4035-b887-bd12a478c9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:53.671780 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a85a20-e9e8-4e69-9a48-760e72a5a041 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:53.672402 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a85a20-e9e8-4e69-9a48-760e72a5a041 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:53.672870 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a85a20-e9e8-4e69-9a48-760e72a5a041 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:53.673210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a85a20-e9e8-4e69-9a48-760e72a5a041 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:53.674796 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 933/1866] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:53.675006 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-69a85a20-e9e8-4e69-9a48-760e72a5a041 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:53.675917 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be5f49-8dde-4835-8e93-c356c563fb71 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:53.676205 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be5f49-8dde-4835-8e93-c356c563fb71 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:53.676477 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be5f49-8dde-4835-8e93-c356c563fb71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:53.703442 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-51be5f49-8dde-4835-8e93-c356c563fb71 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:53.704943 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-51be5f49-8dde-4835-8e93-c356c563fb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:53.713987 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 934/1867] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:53.715290 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bfa393-33cc-49cb-ab90-b4d398351474 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:53.715577 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bfa393-33cc-49cb-ab90-b4d398351474 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:53.715849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bfa393-33cc-49cb-ab90-b4d398351474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:53.735134 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-c9bfa393-33cc-49cb-ab90-b4d398351474 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:53.736336 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-c9bfa393-33cc-49cb-ab90-b4d398351474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:53.744741 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 935/1868] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:53.745877 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bfa393-33cc-49cb-ab90-b4d398351474 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:53.754758 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a85a20-e9e8-4e69-9a48-760e72a5a041 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:53.756024 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bfa393-33cc-49cb-ab90-b4d398351474 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:53.756501 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a85a20-e9e8-4e69-9a48-760e72a5a041 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:53.761116 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd06184-f1a5-44b1-aacb-0ebedcd368bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:53.761710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd06184-f1a5-44b1-aacb-0ebedcd368bf admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:53.761994 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd06184-f1a5-44b1-aacb-0ebedcd368bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:53.762456 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd06184-f1a5-44b1-aacb-0ebedcd368bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:53.762936 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd06184-f1a5-44b1-aacb-0ebedcd368bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:34:53.763244 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd06184-f1a5-44b1-aacb-0ebedcd368bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:53.764832 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd06184-f1a5-44b1-aacb-0ebedcd368bf admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:53.767134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 934/1869] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:53 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:53.767795 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a85a20-e9e8-4e69-9a48-760e72a5a041 service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:53.779005 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 936/1870] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:34:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:34:53.780186 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a85a20-e9e8-4e69-9a48-760e72a5a041 service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:53.783657 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd06184-f1a5-44b1-aacb-0ebedcd368bf admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:53.785068 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e84311c-4096-4f3d-9864-280b4646609a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:53.785732 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84311c-4096-4f3d-9864-280b4646609a admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:53.785963 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84311c-4096-4f3d-9864-280b4646609a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:53.786494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84311c-4096-4f3d-9864-280b4646609a admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:53.786988 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e84311c-4096-4f3d-9864-280b4646609a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:34:53.787293 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e84311c-4096-4f3d-9864-280b4646609a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:53.788873 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e84311c-4096-4f3d-9864-280b4646609a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:53.795425 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd06184-f1a5-44b1-aacb-0ebedcd368bf admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:53.804458 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 935/1871] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:34:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:34:53.805629 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e84311c-4096-4f3d-9864-280b4646609a admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:53.805930 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dad9dcd-5a15-4fe5-b577-b37401367b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:53.806787 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad9dcd-5a15-4fe5-b577-b37401367b4b admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:53.807115 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad9dcd-5a15-4fe5-b577-b37401367b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:53.807507 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad9dcd-5a15-4fe5-b577-b37401367b4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:53.808174 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dad9dcd-5a15-4fe5-b577-b37401367b4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:53.810181 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dad9dcd-5a15-4fe5-b577-b37401367b4b admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:53.817283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e84311c-4096-4f3d-9864-280b4646609a admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:53.823743 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2fc9e8-398e-4d44-a7e4-c0f6e4399253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:53.824602 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2fc9e8-398e-4d44-a7e4-c0f6e4399253 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:53.824738 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2fc9e8-398e-4d44-a7e4-c0f6e4399253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:53.825042 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2fc9e8-398e-4d44-a7e4-c0f6e4399253 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:53.825665 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2fc9e8-398e-4d44-a7e4-c0f6e4399253 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:53.827921 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2fc9e8-398e-4d44-a7e4-c0f6e4399253 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:53.842311 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 937/1872] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:53 2022] POST /identity/v3/projects => generated 387 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:53.844199 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dad9dcd-5a15-4fe5-b577-b37401367b4b admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:53.865992 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:53.869746 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 936/1873] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:53 2022] POST /identity/v3/projects => generated 365 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:34:53.869746 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2fc9e8-398e-4d44-a7e4-c0f6e4399253 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:53.873346 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8c2fc9e8-398e-4d44-a7e4-c0f6e4399253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:53.927671 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dad9dcd-5a15-4fe5-b577-b37401367b4b admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:53.937262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2fc9e8-398e-4d44-a7e4-c0f6e4399253 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:53.941960 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84910f9f-68dc-4fde-9e94-f8b7dd47baf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:53.942653 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84910f9f-68dc-4fde-9e94-f8b7dd47baf0 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:53.942944 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84910f9f-68dc-4fde-9e94-f8b7dd47baf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:53.943227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84910f9f-68dc-4fde-9e94-f8b7dd47baf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:53.945151 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-84910f9f-68dc-4fde-9e94-f8b7dd47baf0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:54.004408 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c33f3-d2fa-4707-9c80-7aa2c4ad07fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:54.005132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c33f3-d2fa-4707-9c80-7aa2c4ad07fe admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:54.005481 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c33f3-d2fa-4707-9c80-7aa2c4ad07fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:54.005825 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c33f3-d2fa-4707-9c80-7aa2c4ad07fe admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:54.006739 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c33f3-d2fa-4707-9c80-7aa2c4ad07fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:54.009497 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c33f3-d2fa-4707-9c80-7aa2c4ad07fe admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:54.019161 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84910f9f-68dc-4fde-9e94-f8b7dd47baf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:54.020755 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84910f9f-68dc-4fde-9e94-f8b7dd47baf0 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:54.046351 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 937/1874] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:54.047205 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84910f9f-68dc-4fde-9e94-f8b7dd47baf0 service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:54.050702 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 938/1875] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:53 2022] POST /identity/v3/users => generated 294 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:34:54.051606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c33f3-d2fa-4707-9c80-7aa2c4ad07fe admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:54.060995 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84910f9f-68dc-4fde-9e94-f8b7dd47baf0 service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:54.063982 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c33f3-d2fa-4707-9c80-7aa2c4ad07fe admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:54.065939 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1721c05-604b-4e1b-bb5b-b1dc12518f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:54.066763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1721c05-604b-4e1b-bb5b-b1dc12518f20 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:54.067133 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1721c05-604b-4e1b-bb5b-b1dc12518f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:54.067502 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1721c05-604b-4e1b-bb5b-b1dc12518f20 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:54.068150 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1721c05-604b-4e1b-bb5b-b1dc12518f20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:54.069673 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf3c3d-4d5e-4aee-9d65-4090955cd800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:54.070404 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf3c3d-4d5e-4aee-9d65-4090955cd800 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:54.070515 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1721c05-604b-4e1b-bb5b-b1dc12518f20 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:54.071219 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf3c3d-4d5e-4aee-9d65-4090955cd800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:54.071506 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf3c3d-4d5e-4aee-9d65-4090955cd800 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:54.072164 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf3c3d-4d5e-4aee-9d65-4090955cd800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:54.075275 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf3c3d-4d5e-4aee-9d65-4090955cd800 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:54.080697 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:54.092177 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 939/1876] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:54.096310 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf3c3d-4d5e-4aee-9d65-4090955cd800 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:54.098446 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 938/1877] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:54 2022] POST /identity/v3/users => generated 283 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:54.100301 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1721c05-604b-4e1b-bb5b-b1dc12518f20 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:54.111647 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf3c3d-4d5e-4aee-9d65-4090955cd800 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:54.114398 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1721c05-604b-4e1b-bb5b-b1dc12518f20 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:54.117323 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47019bc-23b1-4406-aa6f-961e2d7ee40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:54.118103 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47019bc-23b1-4406-aa6f-961e2d7ee40c admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:54.118524 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47019bc-23b1-4406-aa6f-961e2d7ee40c admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:54.118865 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47019bc-23b1-4406-aa6f-961e2d7ee40c admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:54.120222 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47019bc-23b1-4406-aa6f-961e2d7ee40c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:54.122094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47019bc-23b1-4406-aa6f-961e2d7ee40c admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:54.124077 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eea5db-a414-4d85-8ea1-f628c148f0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:54.124704 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eea5db-a414-4d85-8ea1-f628c148f0e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:54.125062 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eea5db-a414-4d85-8ea1-f628c148f0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:54.125255 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eea5db-a414-4d85-8ea1-f628c148f0e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:54.126015 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eea5db-a414-4d85-8ea1-f628c148f0e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:54.127565 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eea5db-a414-4d85-8ea1-f628c148f0e1 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:54.139911 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 939/1878] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:34:54.145804 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 940/1879] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:54.145960 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eea5db-a414-4d85-8ea1-f628c148f0e1 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:54.147288 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47019bc-23b1-4406-aa6f-961e2d7ee40c admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:54.155876 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eea5db-a414-4d85-8ea1-f628c148f0e1 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:54.162167 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47019bc-23b1-4406-aa6f-961e2d7ee40c admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:54.163068 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b48fe9-619a-4687-9d41-8bdf86f2141f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:54.163837 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b48fe9-619a-4687-9d41-8bdf86f2141f admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:54.163950 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b48fe9-619a-4687-9d41-8bdf86f2141f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:54.164220 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b48fe9-619a-4687-9d41-8bdf86f2141f admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:54.164891 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b48fe9-619a-4687-9d41-8bdf86f2141f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:54.166502 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b48fe9-619a-4687-9d41-8bdf86f2141f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:54.167437 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bdba93-afb0-4281-acfb-eebc73a4f027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:54.167901 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bdba93-afb0-4281-acfb-eebc73a4f027 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:54.168362 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bdba93-afb0-4281-acfb-eebc73a4f027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:54.168639 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bdba93-afb0-4281-acfb-eebc73a4f027 admin admin] PATH_INFO: `/v3/projects/59c3fa112874404f8ea2724daa8bd8fd/users/7694a288e1604e119fe75597ebf64152/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:54.179154 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 940/1880] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:34:54.183574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b48fe9-619a-4687-9d41-8bdf86f2141f admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:54.186513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bdba93-afb0-4281-acfb-eebc73a4f027 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59c3fa112874404f8ea2724daa8bd8fd, user_id=7694a288e1604e119fe75597ebf64152, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:54.192144 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bdba93-afb0-4281-acfb-eebc73a4f027 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:54.206197 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b48fe9-619a-4687-9d41-8bdf86f2141f admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:54.213606 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84efdc2a-eaaa-49bd-af2c-cd6fc48dd36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:54.214481 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efdc2a-eaaa-49bd-af2c-cd6fc48dd36b admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:54.215105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efdc2a-eaaa-49bd-af2c-cd6fc48dd36b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:54.215597 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efdc2a-eaaa-49bd-af2c-cd6fc48dd36b admin admin] PATH_INFO: `/v3/projects/a141fb4b221348dab50db9f4ec785788/users/6aed24c9c7134e27bc3c516855ed0960/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:54.221855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 941/1881] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:54 2022] PUT /identity/v3/projects/59c3fa112874404f8ea2724daa8bd8fd/users/7694a288e1604e119fe75597ebf64152/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:54.234736 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f66f9-8d56-46df-b0c3-a677ff1bb4df None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:54.235163 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f66f9-8d56-46df-b0c3-a677ff1bb4df None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:54.235531 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f66f9-8d56-46df-b0c3-a677ff1bb4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:54.235713 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84efdc2a-eaaa-49bd-af2c-cd6fc48dd36b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a141fb4b221348dab50db9f4ec785788, user_id=6aed24c9c7134e27bc3c516855ed0960, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:54.238643 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84efdc2a-eaaa-49bd-af2c-cd6fc48dd36b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:54.272486 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-598f66f9-8d56-46df-b0c3-a677ff1bb4df None None] MFA Rules not processed for user `7694a288e1604e119fe75597ebf64152`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:54.274329 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-598f66f9-8d56-46df-b0c3-a677ff1bb4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:54.287527 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 941/1882] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:54 2022] PUT /identity/v3/projects/a141fb4b221348dab50db9f4ec785788/users/6aed24c9c7134e27bc3c516855ed0960/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:54.298503 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d580e0-306a-475f-880c-4a1f609a0c68 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:54.298791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d580e0-306a-475f-880c-4a1f609a0c68 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:54.299083 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d580e0-306a-475f-880c-4a1f609a0c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:54.330774 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-c0d580e0-306a-475f-880c-4a1f609a0c68 None None] MFA Rules not processed for user `6aed24c9c7134e27bc3c516855ed0960`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:54.332033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c0d580e0-306a-475f-880c-4a1f609a0c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:54.374348 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:54.387777 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 942/1883] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:54 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:54.389064 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f66f9-8d56-46df-b0c3-a677ff1bb4df None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:54.426451 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 942/1884] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:34:54.430001 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d580e0-306a-475f-880c-4a1f609a0c68 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:54.467480 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f66f9-8d56-46df-b0c3-a677ff1bb4df None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:54.473155 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8c901e-4823-4f18-9731-1c1d220cb8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:54.473888 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c901e-4823-4f18-9731-1c1d220cb8ea admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:54.474170 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c901e-4823-4f18-9731-1c1d220cb8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:54.474476 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c901e-4823-4f18-9731-1c1d220cb8ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:54.475119 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8c901e-4823-4f18-9731-1c1d220cb8ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:54.477049 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8c901e-4823-4f18-9731-1c1d220cb8ea admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:54.509167 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 943/1885] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:54 2022] POST /identity/v3/projects => generated 387 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:54.513107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8c901e-4823-4f18-9731-1c1d220cb8ea admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:54.513832 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d580e0-306a-475f-880c-4a1f609a0c68 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:54.556189 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:54.575367 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ecf737-5694-4959-8779-bdefa03783cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:54.575988 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecf737-5694-4959-8779-bdefa03783cd admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:54.576387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecf737-5694-4959-8779-bdefa03783cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:54.576676 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecf737-5694-4959-8779-bdefa03783cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:54.577321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ecf737-5694-4959-8779-bdefa03783cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:54.579417 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ecf737-5694-4959-8779-bdefa03783cd admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:54.583685 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8c901e-4823-4f18-9731-1c1d220cb8ea admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:54.588331 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bfec5c-abbe-4ee9-a870-9a4737e6e2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:54.588938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfec5c-abbe-4ee9-a870-9a4737e6e2d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:54.589223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfec5c-abbe-4ee9-a870-9a4737e6e2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:54.589646 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfec5c-abbe-4ee9-a870-9a4737e6e2d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:54.590315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bfec5c-abbe-4ee9-a870-9a4737e6e2d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:54.592240 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bfec5c-abbe-4ee9-a870-9a4737e6e2d2 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:54.607604 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 943/1886] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:54 2022] POST /identity/v3/projects => generated 365 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:54.611783 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ecf737-5694-4959-8779-bdefa03783cd admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:54.617330 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 944/1887] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:54 2022] POST /identity/v3/users => generated 300 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:34:54.618560 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bfec5c-abbe-4ee9-a870-9a4737e6e2d2 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:54.629895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bfec5c-abbe-4ee9-a870-9a4737e6e2d2 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:54.674984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ecf737-5694-4959-8779-bdefa03783cd admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:54.680219 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d75c6-5b18-4361-bdda-4cf009e28eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:54.680995 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d75c6-5b18-4361-bdda-4cf009e28eb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:54.681348 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d75c6-5b18-4361-bdda-4cf009e28eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:54.681739 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d75c6-5b18-4361-bdda-4cf009e28eb2 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:54.682747 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d75c6-5b18-4361-bdda-4cf009e28eb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:54.685687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d75c6-5b18-4361-bdda-4cf009e28eb2 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:54.689137 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f14323-83f0-43c8-a614-ad69aacef785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:54.689752 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f14323-83f0-43c8-a614-ad69aacef785 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:54.690037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f14323-83f0-43c8-a614-ad69aacef785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:54.690349 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f14323-83f0-43c8-a614-ad69aacef785 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:54.690998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f14323-83f0-43c8-a614-ad69aacef785 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:54.692603 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f14323-83f0-43c8-a614-ad69aacef785 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:54.706825 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 945/1888] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:34:54.707731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f14323-83f0-43c8-a614-ad69aacef785 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:54.710181 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 944/1889] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:54 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:54.714104 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d75c6-5b18-4361-bdda-4cf009e28eb2 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:54.717572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f14323-83f0-43c8-a614-ad69aacef785 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:54.722266 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53238c9-f4b7-4f6b-adf7-df9cc13de583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:54.722873 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53238c9-f4b7-4f6b-adf7-df9cc13de583 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:54.723364 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53238c9-f4b7-4f6b-adf7-df9cc13de583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:54.723599 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53238c9-f4b7-4f6b-adf7-df9cc13de583 admin admin] PATH_INFO: `/v3/projects/1da65317b2ff4bbeb2a84ff4bf31eb6a/users/7b28b5f5f43c46ce8efea1a676ab4b44/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:54.723926 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d75c6-5b18-4361-bdda-4cf009e28eb2 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:54.726692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:54.729176 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b441dd3-156d-4ee4-a6fd-697b875053c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:54.730029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b441dd3-156d-4ee4-a6fd-697b875053c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:54.730352 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b441dd3-156d-4ee4-a6fd-697b875053c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:54.730647 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b441dd3-156d-4ee4-a6fd-697b875053c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:54.731272 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b441dd3-156d-4ee4-a6fd-697b875053c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:54.732896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b441dd3-156d-4ee4-a6fd-697b875053c4 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:54.740751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53238c9-f4b7-4f6b-adf7-df9cc13de583 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1da65317b2ff4bbeb2a84ff4bf31eb6a, user_id=7b28b5f5f43c46ce8efea1a676ab4b44, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:54.743484 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53238c9-f4b7-4f6b-adf7-df9cc13de583 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:54.743883 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 945/1890] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:34:54.748669 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b441dd3-156d-4ee4-a6fd-697b875053c4 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:54.756935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 946/1891] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:54 2022] PUT /identity/v3/projects/1da65317b2ff4bbeb2a84ff4bf31eb6a/users/7b28b5f5f43c46ce8efea1a676ab4b44/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:54.758165 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b441dd3-156d-4ee4-a6fd-697b875053c4 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:54.760637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53238c9-f4b7-4f6b-adf7-df9cc13de583 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:54.815004 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b414c-fefb-4f4e-8c4a-fe9cb0b4260b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:54.815662 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b414c-fefb-4f4e-8c4a-fe9cb0b4260b admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:54.815942 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b414c-fefb-4f4e-8c4a-fe9cb0b4260b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:54.816212 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b414c-fefb-4f4e-8c4a-fe9cb0b4260b admin admin] PATH_INFO: `/v3/projects/40b47ea752e14a1c97fa0a540dbac92f/users/4a798d5ed5cc458c8c88472bb10dc366/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:54.821400 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53238c9-f4b7-4f6b-adf7-df9cc13de583 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:54.826264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e68fa0-e7b6-4a75-a20c-64323538be9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:54.826880 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e68fa0-e7b6-4a75-a20c-64323538be9d admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:54.827163 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e68fa0-e7b6-4a75-a20c-64323538be9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:54.827575 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e68fa0-e7b6-4a75-a20c-64323538be9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:54.828179 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e68fa0-e7b6-4a75-a20c-64323538be9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:54.829838 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e68fa0-e7b6-4a75-a20c-64323538be9d admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:54.830804 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b414c-fefb-4f4e-8c4a-fe9cb0b4260b admin admin] RBAC: Authorizing `identity:create_grant(project_id=40b47ea752e14a1c97fa0a540dbac92f, user_id=4a798d5ed5cc458c8c88472bb10dc366, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:54.833457 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b414c-fefb-4f4e-8c4a-fe9cb0b4260b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:54.841621 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 947/1892] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:34:54.845769 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e68fa0-e7b6-4a75-a20c-64323538be9d admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:54.847924 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 946/1893] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:54 2022] PUT /identity/v3/projects/40b47ea752e14a1c97fa0a540dbac92f/users/4a798d5ed5cc458c8c88472bb10dc366/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:54.851552 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b414c-fefb-4f4e-8c4a-fe9cb0b4260b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:54.856570 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e68fa0-e7b6-4a75-a20c-64323538be9d admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:54.882197 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:54.914258 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167bfea5-b5d1-4050-b8d1-af707ee71180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:54.914935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167bfea5-b5d1-4050-b8d1-af707ee71180 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:54.915217 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167bfea5-b5d1-4050-b8d1-af707ee71180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:54.915494 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167bfea5-b5d1-4050-b8d1-af707ee71180 admin admin] PATH_INFO: `/v3/projects/1da65317b2ff4bbeb2a84ff4bf31eb6a/users/7b28b5f5f43c46ce8efea1a676ab4b44/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:54.918141 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167bfea5-b5d1-4050-b8d1-af707ee71180 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1da65317b2ff4bbeb2a84ff4bf31eb6a, user_id=7b28b5f5f43c46ce8efea1a676ab4b44, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:54.918313 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b414c-fefb-4f4e-8c4a-fe9cb0b4260b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:54.920850 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167bfea5-b5d1-4050-b8d1-af707ee71180 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:54.922986 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdbfe1-ec92-4862-9977-c0b30cc50878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:54.923619 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdbfe1-ec92-4862-9977-c0b30cc50878 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:54.923901 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdbfe1-ec92-4862-9977-c0b30cc50878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:54.924180 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdbfe1-ec92-4862-9977-c0b30cc50878 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:54.925081 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccdbfe1-ec92-4862-9977-c0b30cc50878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:54.926868 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccdbfe1-ec92-4862-9977-c0b30cc50878 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:54.934215 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 948/1894] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:54 2022] PUT /identity/v3/projects/1da65317b2ff4bbeb2a84ff4bf31eb6a/users/7b28b5f5f43c46ce8efea1a676ab4b44/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:54.942957 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 947/1895] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:54.943400 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167bfea5-b5d1-4050-b8d1-af707ee71180 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:54.944125 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db224e7-982f-4c91-8f19-759bba1f0479 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:54.944436 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db224e7-982f-4c91-8f19-759bba1f0479 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:54.944717 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db224e7-982f-4c91-8f19-759bba1f0479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:54.972233 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-8db224e7-982f-4c91-8f19-759bba1f0479 None None] MFA Rules not processed for user `7b28b5f5f43c46ce8efea1a676ab4b44`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:54.973364 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8db224e7-982f-4c91-8f19-759bba1f0479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:55.004750 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167bfea5-b5d1-4050-b8d1-af707ee71180 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:55.009404 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76757b81-7674-4c7d-8f3a-18cc2d241ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:55.010173 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76757b81-7674-4c7d-8f3a-18cc2d241ec3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:55.010488 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76757b81-7674-4c7d-8f3a-18cc2d241ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:55.010770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76757b81-7674-4c7d-8f3a-18cc2d241ec3 admin admin] PATH_INFO: `/v3/projects/40b47ea752e14a1c97fa0a540dbac92f/users/4a798d5ed5cc458c8c88472bb10dc366/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:55.012761 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76757b81-7674-4c7d-8f3a-18cc2d241ec3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40b47ea752e14a1c97fa0a540dbac92f, user_id=4a798d5ed5cc458c8c88472bb10dc366, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:55.015522 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76757b81-7674-4c7d-8f3a-18cc2d241ec3 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:55.026434 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 949/1896] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:54 2022] PUT /identity/v3/projects/40b47ea752e14a1c97fa0a540dbac92f/users/4a798d5ed5cc458c8c88472bb10dc366/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:55.038018 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a7853-9682-4842-87b5-04f1b77779f5 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:55.038514 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a7853-9682-4842-87b5-04f1b77779f5 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:55.038804 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a7853-9682-4842-87b5-04f1b77779f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:55.052279 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 948/1897] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:54 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:34:55.059756 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db224e7-982f-4c91-8f19-759bba1f0479 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:55.061235 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8db224e7-982f-4c91-8f19-759bba1f0479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:55.073815 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:55.083980 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-946a7853-9682-4842-87b5-04f1b77779f5 None None] MFA Rules not processed for user `4a798d5ed5cc458c8c88472bb10dc366`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:55.085827 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-946a7853-9682-4842-87b5-04f1b77779f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:55.124816 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db224e7-982f-4c91-8f19-759bba1f0479 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:55.166070 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 950/1898] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:55 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:55.179359 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a7853-9682-4842-87b5-04f1b77779f5 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:55.180815 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190aa62-39bc-4f36-9ce9-28dc01d9aeec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:55.181525 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190aa62-39bc-4f36-9ce9-28dc01d9aeec service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:55.181877 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190aa62-39bc-4f36-9ce9-28dc01d9aeec service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:55.182119 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190aa62-39bc-4f36-9ce9-28dc01d9aeec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:55.191229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a7853-9682-4842-87b5-04f1b77779f5 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:55.195352 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1defd-84b0-49f9-9ce0-86fd530c7941 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:55.196062 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1defd-84b0-49f9-9ce0-86fd530c7941 service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:55.196365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1defd-84b0-49f9-9ce0-86fd530c7941 service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:55.196642 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1defd-84b0-49f9-9ce0-86fd530c7941 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:55.203790 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:55.206984 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1defd-84b0-49f9-9ce0-86fd530c7941 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:55.208512 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1defd-84b0-49f9-9ce0-86fd530c7941 service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:55.220045 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 951/1899] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:55 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:34:55.271500 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190aa62-39bc-4f36-9ce9-28dc01d9aeec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:55.272958 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190aa62-39bc-4f36-9ce9-28dc01d9aeec service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:55.283770 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 949/1900] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:55 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:55.318032 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1defd-84b0-49f9-9ce0-86fd530c7941 service cinder] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:55.332097 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190aa62-39bc-4f36-9ce9-28dc01d9aeec service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:55.344398 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190aa62-39bc-4f36-9ce9-28dc01d9aeec service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:55.349745 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a84e4-e8ed-4bc9-ae0a-7deb8614fcbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:55.350663 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a84e4-e8ed-4bc9-ae0a-7deb8614fcbe service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:55.351152 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a84e4-e8ed-4bc9-ae0a-7deb8614fcbe service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:55.351629 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a84e4-e8ed-4bc9-ae0a-7deb8614fcbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:55.409395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1defd-84b0-49f9-9ce0-86fd530c7941 service cinder] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:55.413970 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5862624d-fdb4-49ac-bdd6-39f63e81b92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:55.414642 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862624d-fdb4-49ac-bdd6-39f63e81b92b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:55.414925 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862624d-fdb4-49ac-bdd6-39f63e81b92b admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:55.415199 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862624d-fdb4-49ac-bdd6-39f63e81b92b admin admin] PATH_INFO: `/v3/users/7694a288e1604e119fe75597ebf64152` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:55.416865 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5862624d-fdb4-49ac-bdd6-39f63e81b92b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7694a288e1604e119fe75597ebf64152)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:55.418816 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5862624d-fdb4-49ac-bdd6-39f63e81b92b admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:55.422089 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a84e4-e8ed-4bc9-ae0a-7deb8614fcbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:55.423560 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a84e4-e8ed-4bc9-ae0a-7deb8614fcbe service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:55.433298 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 950/1901] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:34:55.465984 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5862624d-fdb4-49ac-bdd6-39f63e81b92b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7694a288e1604e119fe75597ebf64152'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:55.466746 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5862624d-fdb4-49ac-bdd6-39f63e81b92b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7694a288e1604e119fe75597ebf64152'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:55.476427 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5862624d-fdb4-49ac-bdd6-39f63e81b92b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7694a288e1604e119fe75597ebf64152'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:55.477170 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5862624d-fdb4-49ac-bdd6-39f63e81b92b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7694a288e1604e119fe75597ebf64152'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:55.493087 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5862624d-fdb4-49ac-bdd6-39f63e81b92b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7694a288e1604e119fe75597ebf64152'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:55.502619 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:55.507218 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 952/1902] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:55 2022] DELETE /identity/v3/users/7694a288e1604e119fe75597ebf64152 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:55.512202 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c91223-fe32-4c6e-a48e-ace084c4a818 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:55.512481 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c91223-fe32-4c6e-a48e-ace084c4a818 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:55.512771 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c91223-fe32-4c6e-a48e-ace084c4a818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:55.544169 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-18c91223-fe32-4c6e-a48e-ace084c4a818 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:55.545973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-18c91223-fe32-4c6e-a48e-ace084c4a818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:55.660809 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 951/1903] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:34:55.668703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5862624d-fdb4-49ac-bdd6-39f63e81b92b admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:55.670025 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-5862624d-fdb4-49ac-bdd6-39f63e81b92b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:55.750617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5862624d-fdb4-49ac-bdd6-39f63e81b92b admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:55.754883 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9245e-d08e-459d-9d61-8a0cd618363c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:55.756186 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9245e-d08e-459d-9d61-8a0cd618363c service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:55.756186 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9245e-d08e-459d-9d61-8a0cd618363c service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:55.756186 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9245e-d08e-459d-9d61-8a0cd618363c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:55.766384 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a9245e-d08e-459d-9d61-8a0cd618363c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:55.767992 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a9245e-d08e-459d-9d61-8a0cd618363c service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:55.779510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 953/1904] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:56.063457 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c91223-fe32-4c6e-a48e-ace084c4a818 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:56.064912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-18c91223-fe32-4c6e-a48e-ace084c4a818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:56.079063 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c91223-fe32-4c6e-a48e-ace084c4a818 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:56.084586 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c86427-9c7e-4744-b5c2-86e3be99cd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:56.085240 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c86427-9c7e-4744-b5c2-86e3be99cd0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:56.085516 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c86427-9c7e-4744-b5c2-86e3be99cd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:56.085798 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c86427-9c7e-4744-b5c2-86e3be99cd0e admin admin] PATH_INFO: `/v3/projects/59c3fa112874404f8ea2724daa8bd8fd` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:56.086982 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c86427-9c7e-4744-b5c2-86e3be99cd0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=59c3fa112874404f8ea2724daa8bd8fd)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:56.094601 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c86427-9c7e-4744-b5c2-86e3be99cd0e admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:56.152876 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-17c86427-9c7e-4744-b5c2-86e3be99cd0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59c3fa112874404f8ea2724daa8bd8fd', 'request_id': 'req-17c86427-9c7e-4744-b5c2-86e3be99cd0e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:56.154373 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:56.162879 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-17c86427-9c7e-4744-b5c2-86e3be99cd0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59c3fa112874404f8ea2724daa8bd8fd', 'request_id': 'req-17c86427-9c7e-4744-b5c2-86e3be99cd0e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:56.171544 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-17c86427-9c7e-4744-b5c2-86e3be99cd0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59c3fa112874404f8ea2724daa8bd8fd', 'request_id': 'req-17c86427-9c7e-4744-b5c2-86e3be99cd0e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:56.183245 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-17c86427-9c7e-4744-b5c2-86e3be99cd0e admin admin] The token cache is being invalidate because project 59c3fa112874404f8ea2724daa8bd8fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:56.183536 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-17c86427-9c7e-4744-b5c2-86e3be99cd0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:56.184229 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-17c86427-9c7e-4744-b5c2-86e3be99cd0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:56.185763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 952/1905] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:56 2022] DELETE /identity/v3/projects/59c3fa112874404f8ea2724daa8bd8fd => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:56.189666 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9245e-d08e-459d-9d61-8a0cd618363c service neutron] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:56.211615 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9245e-d08e-459d-9d61-8a0cd618363c service neutron] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:56.292177 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29065a7-9180-439f-89b3-36382136eef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:56.293306 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29065a7-9180-439f-89b3-36382136eef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:56.293840 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29065a7-9180-439f-89b3-36382136eef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:56.294165 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29065a7-9180-439f-89b3-36382136eef7 admin admin] PATH_INFO: `/v3/users/7b28b5f5f43c46ce8efea1a676ab4b44` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:56.297287 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29065a7-9180-439f-89b3-36382136eef7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b28b5f5f43c46ce8efea1a676ab4b44)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:56.299635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29065a7-9180-439f-89b3-36382136eef7 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:56.373001 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-a29065a7-9180-439f-89b3-36382136eef7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b28b5f5f43c46ce8efea1a676ab4b44'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:56.373951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-a29065a7-9180-439f-89b3-36382136eef7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b28b5f5f43c46ce8efea1a676ab4b44'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:56.404655 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-a29065a7-9180-439f-89b3-36382136eef7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b28b5f5f43c46ce8efea1a676ab4b44'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:56.407799 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-a29065a7-9180-439f-89b3-36382136eef7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b28b5f5f43c46ce8efea1a676ab4b44'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:56.438960 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-a29065a7-9180-439f-89b3-36382136eef7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b28b5f5f43c46ce8efea1a676ab4b44'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:56.459463 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 954/1906] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:56 2022] DELETE /identity/v3/users/7b28b5f5f43c46ce8efea1a676ab4b44 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:57.010067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c86427-9c7e-4744-b5c2-86e3be99cd0e admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:57.011561 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-17c86427-9c7e-4744-b5c2-86e3be99cd0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:57.034106 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c86427-9c7e-4744-b5c2-86e3be99cd0e admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:57.106064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9d767-f0dd-4093-9305-e26cbb3ebd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:57.106922 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9d767-f0dd-4093-9305-e26cbb3ebd2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:57.107852 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9d767-f0dd-4093-9305-e26cbb3ebd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:57.107852 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9d767-f0dd-4093-9305-e26cbb3ebd2c admin admin] PATH_INFO: `/v3/projects/1da65317b2ff4bbeb2a84ff4bf31eb6a` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:57.108745 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b9d767-f0dd-4093-9305-e26cbb3ebd2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1da65317b2ff4bbeb2a84ff4bf31eb6a)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:57.110877 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b9d767-f0dd-4093-9305-e26cbb3ebd2c admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:57.168000 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e1b9d767-f0dd-4093-9305-e26cbb3ebd2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1da65317b2ff4bbeb2a84ff4bf31eb6a', 'request_id': 'req-e1b9d767-f0dd-4093-9305-e26cbb3ebd2c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:57.174147 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e1b9d767-f0dd-4093-9305-e26cbb3ebd2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1da65317b2ff4bbeb2a84ff4bf31eb6a', 'request_id': 'req-e1b9d767-f0dd-4093-9305-e26cbb3ebd2c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:57.183137 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e1b9d767-f0dd-4093-9305-e26cbb3ebd2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1da65317b2ff4bbeb2a84ff4bf31eb6a', 'request_id': 'req-e1b9d767-f0dd-4093-9305-e26cbb3ebd2c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:57.193897 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e1b9d767-f0dd-4093-9305-e26cbb3ebd2c admin admin] The token cache is being invalidate because project 1da65317b2ff4bbeb2a84ff4bf31eb6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:57.194217 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e1b9d767-f0dd-4093-9305-e26cbb3ebd2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:57.194869 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e1b9d767-f0dd-4093-9305-e26cbb3ebd2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:57.196347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 953/1907] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:57 2022] DELETE /identity/v3/projects/1da65317b2ff4bbeb2a84ff4bf31eb6a => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:57.201238 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea062c4-5215-49e7-90b3-48cc3b139c7b None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:57.201699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea062c4-5215-49e7-90b3-48cc3b139c7b None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:57.201805 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea062c4-5215-49e7-90b3-48cc3b139c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:57.225230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-aea062c4-5215-49e7-90b3-48cc3b139c7b None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:57.226499 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-aea062c4-5215-49e7-90b3-48cc3b139c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:57.245338 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:57.291697 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 955/1908] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:57.299344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb485686-058c-4a4f-93f2-3ca226e60ef0 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:57.299630 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb485686-058c-4a4f-93f2-3ca226e60ef0 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:57.299907 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb485686-058c-4a4f-93f2-3ca226e60ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:57.319784 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-bb485686-058c-4a4f-93f2-3ca226e60ef0 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:57.320994 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-bb485686-058c-4a4f-93f2-3ca226e60ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:57.328734 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 954/1909] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:57.332893 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea062c4-5215-49e7-90b3-48cc3b139c7b None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:57.343003 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea062c4-5215-49e7-90b3-48cc3b139c7b None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:57.347578 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30ab9b-ce41-42be-812d-c684161a1ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:57.348188 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30ab9b-ce41-42be-812d-c684161a1ea6 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:57.348473 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30ab9b-ce41-42be-812d-c684161a1ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:57.348750 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30ab9b-ce41-42be-812d-c684161a1ea6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:57.349397 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a30ab9b-ce41-42be-812d-c684161a1ea6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:34:57.349700 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a30ab9b-ce41-42be-812d-c684161a1ea6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:57.351342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a30ab9b-ce41-42be-812d-c684161a1ea6 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:57.364029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 956/1910] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:34:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:34:57.368179 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb485686-058c-4a4f-93f2-3ca226e60ef0 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:57.377588 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb485686-058c-4a4f-93f2-3ca226e60ef0 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:57.382257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20190a8-3ab9-4351-90c0-10cbc2162f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:57.382872 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20190a8-3ab9-4351-90c0-10cbc2162f1b admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:57.383151 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20190a8-3ab9-4351-90c0-10cbc2162f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:57.383421 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20190a8-3ab9-4351-90c0-10cbc2162f1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:57.384047 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20190a8-3ab9-4351-90c0-10cbc2162f1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:57.385992 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20190a8-3ab9-4351-90c0-10cbc2162f1b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:57.412907 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 955/1911] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:57 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:57.418177 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30ab9b-ce41-42be-812d-c684161a1ea6 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:57.428264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30ab9b-ce41-42be-812d-c684161a1ea6 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:57.484721 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22b7c1-1b4a-408c-83a0-72fc45972e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:57.485331 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22b7c1-1b4a-408c-83a0-72fc45972e07 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:57.485617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22b7c1-1b4a-408c-83a0-72fc45972e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:57.485895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22b7c1-1b4a-408c-83a0-72fc45972e07 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:57.486576 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a22b7c1-1b4a-408c-83a0-72fc45972e07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:57.488545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a22b7c1-1b4a-408c-83a0-72fc45972e07 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:57.514507 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 957/1912] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:57 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:57.515689 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20190a8-3ab9-4351-90c0-10cbc2162f1b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:57.526154 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20190a8-3ab9-4351-90c0-10cbc2162f1b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:57.531462 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7a79c-640b-4ab5-af51-491f87a6821c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:57.532197 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7a79c-640b-4ab5-af51-491f87a6821c admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:57.532573 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7a79c-640b-4ab5-af51-491f87a6821c admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:57.532939 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7a79c-640b-4ab5-af51-491f87a6821c admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:57.533732 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7a79c-640b-4ab5-af51-491f87a6821c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:57.535782 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7a79c-640b-4ab5-af51-491f87a6821c admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:57.548517 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 956/1913] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:34:57.552350 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22b7c1-1b4a-408c-83a0-72fc45972e07 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:57.562479 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22b7c1-1b4a-408c-83a0-72fc45972e07 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:57.567078 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb956e2-5db4-439c-8025-2bcf1a0b7480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:57.567684 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb956e2-5db4-439c-8025-2bcf1a0b7480 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:57.567973 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb956e2-5db4-439c-8025-2bcf1a0b7480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:57.568259 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb956e2-5db4-439c-8025-2bcf1a0b7480 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:57.568937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb956e2-5db4-439c-8025-2bcf1a0b7480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:57.570563 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb956e2-5db4-439c-8025-2bcf1a0b7480 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:57.582098 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 958/1914] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:57.586552 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7a79c-640b-4ab5-af51-491f87a6821c admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:57.597027 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7a79c-640b-4ab5-af51-491f87a6821c admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:57.601669 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202cb257-5b7a-40a1-b246-3140117dcb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:57.602357 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202cb257-5b7a-40a1-b246-3140117dcb77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:57.602644 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202cb257-5b7a-40a1-b246-3140117dcb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:57.602920 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202cb257-5b7a-40a1-b246-3140117dcb77 admin admin] PATH_INFO: `/v3/projects/95a069d223ab4bcfaca261894b6fc284/users/ccb072bf127947d4bf5cd19ec9d860af/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:57.617982 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202cb257-5b7a-40a1-b246-3140117dcb77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95a069d223ab4bcfaca261894b6fc284, user_id=ccb072bf127947d4bf5cd19ec9d860af, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:57.620750 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202cb257-5b7a-40a1-b246-3140117dcb77 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:57.635953 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 957/1915] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:57 2022] PUT /identity/v3/projects/95a069d223ab4bcfaca261894b6fc284/users/ccb072bf127947d4bf5cd19ec9d860af/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:57.647649 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e893cc-ed29-4b31-b888-30d33306d929 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:57.648648 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e893cc-ed29-4b31-b888-30d33306d929 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:57.649050 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e893cc-ed29-4b31-b888-30d33306d929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:57.676918 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-e3e893cc-ed29-4b31-b888-30d33306d929 None None] MFA Rules not processed for user `ccb072bf127947d4bf5cd19ec9d860af`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:57.678129 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-e3e893cc-ed29-4b31-b888-30d33306d929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:57.742601 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 959/1916] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:57.747221 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202cb257-5b7a-40a1-b246-3140117dcb77 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:57.753670 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:57.813881 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202cb257-5b7a-40a1-b246-3140117dcb77 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:57.818676 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a47c7c-cbb8-4d9e-856a-23e53046574f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:57.819299 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a47c7c-cbb8-4d9e-856a-23e53046574f admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:57.819595 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a47c7c-cbb8-4d9e-856a-23e53046574f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:57.819868 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a47c7c-cbb8-4d9e-856a-23e53046574f admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:57.820501 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a47c7c-cbb8-4d9e-856a-23e53046574f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:57.823041 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a47c7c-cbb8-4d9e-856a-23e53046574f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:57.850902 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 958/1917] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:57 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:34:57.855115 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e893cc-ed29-4b31-b888-30d33306d929 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:57.865961 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e893cc-ed29-4b31-b888-30d33306d929 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:57.879585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:57.959298 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d988f-61c6-4dd3-8c95-ea1b10c4bc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:57.959918 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d988f-61c6-4dd3-8c95-ea1b10c4bc1a admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:57.960213 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d988f-61c6-4dd3-8c95-ea1b10c4bc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:57.960478 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d988f-61c6-4dd3-8c95-ea1b10c4bc1a admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:57.961117 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5d988f-61c6-4dd3-8c95-ea1b10c4bc1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:57.963083 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5d988f-61c6-4dd3-8c95-ea1b10c4bc1a admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:57.985884 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 960/1918] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:57 2022] POST /identity/v3/users => generated 297 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:57.989687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a47c7c-cbb8-4d9e-856a-23e53046574f admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:58.000384 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a47c7c-cbb8-4d9e-856a-23e53046574f admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:58.005139 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e76b0-69b5-45ea-bcdf-34042bc320b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:58.005763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e76b0-69b5-45ea-bcdf-34042bc320b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:58.006050 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e76b0-69b5-45ea-bcdf-34042bc320b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:58.006358 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e76b0-69b5-45ea-bcdf-34042bc320b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:58.007009 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03e76b0-69b5-45ea-bcdf-34042bc320b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:58.008812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03e76b0-69b5-45ea-bcdf-34042bc320b2 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:58.024070 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 959/1919] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:58.028130 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d988f-61c6-4dd3-8c95-ea1b10c4bc1a admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:58.039212 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d988f-61c6-4dd3-8c95-ea1b10c4bc1a admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:58.044445 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d42403-8524-42ed-8b3d-416aeab362f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:58.045371 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d42403-8524-42ed-8b3d-416aeab362f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:58.045661 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d42403-8524-42ed-8b3d-416aeab362f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:58.046067 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d42403-8524-42ed-8b3d-416aeab362f4 admin admin] PATH_INFO: `/v3/projects/830ff35da08849a29c3ce63880466d1c/users/6b6a51e2bc02484e99c719d255385848/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:58.063229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d42403-8524-42ed-8b3d-416aeab362f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=830ff35da08849a29c3ce63880466d1c, user_id=6b6a51e2bc02484e99c719d255385848, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:58.066493 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d42403-8524-42ed-8b3d-416aeab362f4 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:58.082761 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 961/1920] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:58 2022] PUT /identity/v3/projects/830ff35da08849a29c3ce63880466d1c/users/6b6a51e2bc02484e99c719d255385848/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:58.086882 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e76b0-69b5-45ea-bcdf-34042bc320b2 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:58.096687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e76b0-69b5-45ea-bcdf-34042bc320b2 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:58.157218 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33ed54-e181-4181-9748-9d91e4155ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:58.157855 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33ed54-e181-4181-9748-9d91e4155ec1 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:58.158130 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33ed54-e181-4181-9748-9d91e4155ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:58.158430 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33ed54-e181-4181-9748-9d91e4155ec1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:58.159714 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e33ed54-e181-4181-9748-9d91e4155ec1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:58.160685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e33ed54-e181-4181-9748-9d91e4155ec1 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:58.171929 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 960/1921] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:58.175967 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d42403-8524-42ed-8b3d-416aeab362f4 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:58.186775 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d42403-8524-42ed-8b3d-416aeab362f4 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:58.191379 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8913d53a-6b7a-4e55-a705-a0952738e23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:58.192028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913d53a-6b7a-4e55-a705-a0952738e23f admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:58.192315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913d53a-6b7a-4e55-a705-a0952738e23f admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:58.192594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913d53a-6b7a-4e55-a705-a0952738e23f admin admin] PATH_INFO: `/v3/projects/830ff35da08849a29c3ce63880466d1c/users/6b6a51e2bc02484e99c719d255385848/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:58.194505 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8913d53a-6b7a-4e55-a705-a0952738e23f admin admin] RBAC: Authorizing `identity:create_grant(project_id=830ff35da08849a29c3ce63880466d1c, user_id=6b6a51e2bc02484e99c719d255385848, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:58.197158 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8913d53a-6b7a-4e55-a705-a0952738e23f admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:58.207420 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 962/1922] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:58 2022] PUT /identity/v3/projects/830ff35da08849a29c3ce63880466d1c/users/6b6a51e2bc02484e99c719d255385848/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:58.217303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb70314-184a-4707-94ed-4d499381c81c None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:58.217598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb70314-184a-4707-94ed-4d499381c81c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:58.217878 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb70314-184a-4707-94ed-4d499381c81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:58.246259 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-deb70314-184a-4707-94ed-4d499381c81c None None] MFA Rules not processed for user `6b6a51e2bc02484e99c719d255385848`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:58.247597 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-deb70314-184a-4707-94ed-4d499381c81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:58.315045 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:58.319296 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 961/1923] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:34:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:58.329747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8913d53a-6b7a-4e55-a705-a0952738e23f admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:58.331533 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-8913d53a-6b7a-4e55-a705-a0952738e23f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:58.389970 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8913d53a-6b7a-4e55-a705-a0952738e23f admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:58.393620 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b72d90-1f4d-4561-a11f-0fb5c143d345 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:58.394255 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b72d90-1f4d-4561-a11f-0fb5c143d345 service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:58.394571 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b72d90-1f4d-4561-a11f-0fb5c143d345 service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:58.394846 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b72d90-1f4d-4561-a11f-0fb5c143d345 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:58.403745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b72d90-1f4d-4561-a11f-0fb5c143d345 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:58.405193 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b72d90-1f4d-4561-a11f-0fb5c143d345 service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:58.414496 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 963/1924] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:58.517906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb70314-184a-4707-94ed-4d499381c81c None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:58.527326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb70314-184a-4707-94ed-4d499381c81c None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:58.530837 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35625e1a-c3d9-471c-b3f6-7843cc25f747 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:58.531442 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35625e1a-c3d9-471c-b3f6-7843cc25f747 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:58.531720 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35625e1a-c3d9-471c-b3f6-7843cc25f747 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:58.531994 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35625e1a-c3d9-471c-b3f6-7843cc25f747 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:58.589321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35625e1a-c3d9-471c-b3f6-7843cc25f747 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:58.590753 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35625e1a-c3d9-471c-b3f6-7843cc25f747 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:58.599463 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 962/1925] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:01.269841 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b72d90-1f4d-4561-a11f-0fb5c143d345 service cinder] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:01.352653 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b72d90-1f4d-4561-a11f-0fb5c143d345 service cinder] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:01.358928 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4961a2c-e40f-4646-8baa-4502a25438aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:01.360405 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4961a2c-e40f-4646-8baa-4502a25438aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:01.360846 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4961a2c-e40f-4646-8baa-4502a25438aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:01.361281 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4961a2c-e40f-4646-8baa-4502a25438aa admin admin] PATH_INFO: `/v3/users/ccb072bf127947d4bf5cd19ec9d860af` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:01.363306 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4961a2c-e40f-4646-8baa-4502a25438aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccb072bf127947d4bf5cd19ec9d860af)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:01.365347 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4961a2c-e40f-4646-8baa-4502a25438aa admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:01.385617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:01.426790 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-e4961a2c-e40f-4646-8baa-4502a25438aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccb072bf127947d4bf5cd19ec9d860af'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:01.427461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-e4961a2c-e40f-4646-8baa-4502a25438aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccb072bf127947d4bf5cd19ec9d860af'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:01.440283 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-e4961a2c-e40f-4646-8baa-4502a25438aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccb072bf127947d4bf5cd19ec9d860af'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:01.440982 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-e4961a2c-e40f-4646-8baa-4502a25438aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccb072bf127947d4bf5cd19ec9d860af'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:01.467611 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-e4961a2c-e40f-4646-8baa-4502a25438aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccb072bf127947d4bf5cd19ec9d860af'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:01.491413 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 964/1926] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:01 2022] DELETE /identity/v3/users/ccb072bf127947d4bf5cd19ec9d860af => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:01.493043 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a26fe3-7ebe-4edf-9667-38635dccc011 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:01.493499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a26fe3-7ebe-4edf-9667-38635dccc011 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:01.493681 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a26fe3-7ebe-4edf-9667-38635dccc011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:01.513906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-a5a26fe3-7ebe-4edf-9667-38635dccc011 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:01.515290 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a5a26fe3-7ebe-4edf-9667-38635dccc011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:01.598992 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 963/1927] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:01.608809 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4961a2c-e40f-4646-8baa-4502a25438aa admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:01.610479 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-e4961a2c-e40f-4646-8baa-4502a25438aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:01.683692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4961a2c-e40f-4646-8baa-4502a25438aa admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:01.687628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec6aa1-2059-45ff-8308-a2cfe3ad154a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:01.688337 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec6aa1-2059-45ff-8308-a2cfe3ad154a service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:01.688677 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec6aa1-2059-45ff-8308-a2cfe3ad154a service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:01.689065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec6aa1-2059-45ff-8308-a2cfe3ad154a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:01.701886 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec6aa1-2059-45ff-8308-a2cfe3ad154a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:01.703800 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec6aa1-2059-45ff-8308-a2cfe3ad154a service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:01.716988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 965/1928] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:02.112499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a26fe3-7ebe-4edf-9667-38635dccc011 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:02.114535 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a5a26fe3-7ebe-4edf-9667-38635dccc011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:02.131000 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a26fe3-7ebe-4edf-9667-38635dccc011 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:02.137454 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d5b2d-7544-4089-ba0a-7a2b3ca59b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:02.138496 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d5b2d-7544-4089-ba0a-7a2b3ca59b67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:02.139196 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d5b2d-7544-4089-ba0a-7a2b3ca59b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:02.140026 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d5b2d-7544-4089-ba0a-7a2b3ca59b67 admin admin] PATH_INFO: `/v3/projects/95a069d223ab4bcfaca261894b6fc284` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:02.141041 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d5b2d-7544-4089-ba0a-7a2b3ca59b67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95a069d223ab4bcfaca261894b6fc284)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:02.143065 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d5b2d-7544-4089-ba0a-7a2b3ca59b67 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:02.192837 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-6e0d5b2d-7544-4089-ba0a-7a2b3ca59b67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95a069d223ab4bcfaca261894b6fc284', 'request_id': 'req-6e0d5b2d-7544-4089-ba0a-7a2b3ca59b67'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:02.199552 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-6e0d5b2d-7544-4089-ba0a-7a2b3ca59b67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95a069d223ab4bcfaca261894b6fc284', 'request_id': 'req-6e0d5b2d-7544-4089-ba0a-7a2b3ca59b67'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:02.208425 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-6e0d5b2d-7544-4089-ba0a-7a2b3ca59b67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95a069d223ab4bcfaca261894b6fc284', 'request_id': 'req-6e0d5b2d-7544-4089-ba0a-7a2b3ca59b67'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:02.220171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-6e0d5b2d-7544-4089-ba0a-7a2b3ca59b67 admin admin] The token cache is being invalidate because project 95a069d223ab4bcfaca261894b6fc284 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:02.220446 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-6e0d5b2d-7544-4089-ba0a-7a2b3ca59b67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:02.221111 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-6e0d5b2d-7544-4089-ba0a-7a2b3ca59b67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:02.222872 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 964/1929] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:02 2022] DELETE /identity/v3/projects/95a069d223ab4bcfaca261894b6fc284 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:02.226874 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec6aa1-2059-45ff-8308-a2cfe3ad154a service neutron] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:02.237552 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec6aa1-2059-45ff-8308-a2cfe3ad154a service neutron] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:02.297830 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357058c7-534a-43fb-8508-6e47a63c48ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:02.298609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357058c7-534a-43fb-8508-6e47a63c48ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:02.298763 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357058c7-534a-43fb-8508-6e47a63c48ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:02.299037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357058c7-534a-43fb-8508-6e47a63c48ed admin admin] PATH_INFO: `/v3/users/6b6a51e2bc02484e99c719d255385848` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:02.300201 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357058c7-534a-43fb-8508-6e47a63c48ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b6a51e2bc02484e99c719d255385848)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:02.302150 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357058c7-534a-43fb-8508-6e47a63c48ed admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:02.341849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:02.353127 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-357058c7-534a-43fb-8508-6e47a63c48ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b6a51e2bc02484e99c719d255385848'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:02.355202 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-357058c7-534a-43fb-8508-6e47a63c48ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b6a51e2bc02484e99c719d255385848'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:02.363261 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-357058c7-534a-43fb-8508-6e47a63c48ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b6a51e2bc02484e99c719d255385848'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:02.369480 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-357058c7-534a-43fb-8508-6e47a63c48ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b6a51e2bc02484e99c719d255385848'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:02.382390 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-357058c7-534a-43fb-8508-6e47a63c48ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b6a51e2bc02484e99c719d255385848'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:02.396203 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 966/1930] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:02 2022] DELETE /identity/v3/users/6b6a51e2bc02484e99c719d255385848 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:02.672691 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d5b2d-7544-4089-ba0a-7a2b3ca59b67 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:02.674336 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6e0d5b2d-7544-4089-ba0a-7a2b3ca59b67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:02.700469 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d5b2d-7544-4089-ba0a-7a2b3ca59b67 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:02.720658 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:02.770266 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3a8c4-1a26-4047-9e48-590668ca7a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:02.771089 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3a8c4-1a26-4047-9e48-590668ca7a7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:02.771443 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3a8c4-1a26-4047-9e48-590668ca7a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:02.771787 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3a8c4-1a26-4047-9e48-590668ca7a7a admin admin] PATH_INFO: `/v3/projects/830ff35da08849a29c3ce63880466d1c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:02.773390 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3a8c4-1a26-4047-9e48-590668ca7a7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=830ff35da08849a29c3ce63880466d1c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:02.778191 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3a8c4-1a26-4047-9e48-590668ca7a7a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:02.834335 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-cbe3a8c4-1a26-4047-9e48-590668ca7a7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '830ff35da08849a29c3ce63880466d1c', 'request_id': 'req-cbe3a8c4-1a26-4047-9e48-590668ca7a7a'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:02.841264 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-cbe3a8c4-1a26-4047-9e48-590668ca7a7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '830ff35da08849a29c3ce63880466d1c', 'request_id': 'req-cbe3a8c4-1a26-4047-9e48-590668ca7a7a'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:02.849370 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-cbe3a8c4-1a26-4047-9e48-590668ca7a7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '830ff35da08849a29c3ce63880466d1c', 'request_id': 'req-cbe3a8c4-1a26-4047-9e48-590668ca7a7a'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:02.865004 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-cbe3a8c4-1a26-4047-9e48-590668ca7a7a admin admin] The token cache is being invalidate because project 830ff35da08849a29c3ce63880466d1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:02.865836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-cbe3a8c4-1a26-4047-9e48-590668ca7a7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:02.866724 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-cbe3a8c4-1a26-4047-9e48-590668ca7a7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:02.868270 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 965/1931] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:02 2022] DELETE /identity/v3/projects/830ff35da08849a29c3ce63880466d1c => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:02.873510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b9b1a-881d-4b0e-a198-2a10f2d584da None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:02.873802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b9b1a-881d-4b0e-a198-2a10f2d584da None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:02.874080 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b9b1a-881d-4b0e-a198-2a10f2d584da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:02.895523 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-fa1b9b1a-881d-4b0e-a198-2a10f2d584da None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:02.896719 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-fa1b9b1a-881d-4b0e-a198-2a10f2d584da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:03.009132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 967/1932] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:35:03.019627 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32cc640-85f9-45cf-916e-88e07224cea1 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:03.019627 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32cc640-85f9-45cf-916e-88e07224cea1 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:03.027917 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32cc640-85f9-45cf-916e-88e07224cea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:03.075037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-a32cc640-85f9-45cf-916e-88e07224cea1 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:03.076268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a32cc640-85f9-45cf-916e-88e07224cea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:03.097132 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 966/1933] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:03.102335 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b9b1a-881d-4b0e-a198-2a10f2d584da None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:03.113134 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b9b1a-881d-4b0e-a198-2a10f2d584da None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:03.118401 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00ae8a-677c-4ec2-8a41-4c9e8127e893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:03.119544 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00ae8a-677c-4ec2-8a41-4c9e8127e893 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:03.119824 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00ae8a-677c-4ec2-8a41-4c9e8127e893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:03.121539 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00ae8a-677c-4ec2-8a41-4c9e8127e893 admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:03.122220 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae00ae8a-677c-4ec2-8a41-4c9e8127e893 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:35:03.122549 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae00ae8a-677c-4ec2-8a41-4c9e8127e893 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:03.124236 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae00ae8a-677c-4ec2-8a41-4c9e8127e893 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:03.137691 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 968/1934] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:35:03.142746 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32cc640-85f9-45cf-916e-88e07224cea1 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:03.152330 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32cc640-85f9-45cf-916e-88e07224cea1 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:03.157023 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6fa02-1e84-49e9-984f-701abc533198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:03.157738 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6fa02-1e84-49e9-984f-701abc533198 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:03.158118 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6fa02-1e84-49e9-984f-701abc533198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:03.158522 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6fa02-1e84-49e9-984f-701abc533198 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:03.159347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e6fa02-1e84-49e9-984f-701abc533198 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:03.161472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e6fa02-1e84-49e9-984f-701abc533198 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:03.195022 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 967/1935] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:03 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:35:03.196726 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00ae8a-677c-4ec2-8a41-4c9e8127e893 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:03.208258 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00ae8a-677c-4ec2-8a41-4c9e8127e893 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:03.273217 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd0dc32-c752-479a-b998-85a6c5377a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:03.273748 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0dc32-c752-479a-b998-85a6c5377a19 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:03.274027 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0dc32-c752-479a-b998-85a6c5377a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:03.274378 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0dc32-c752-479a-b998-85a6c5377a19 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:03.275067 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd0dc32-c752-479a-b998-85a6c5377a19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:03.277080 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd0dc32-c752-479a-b998-85a6c5377a19 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:03.300976 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 969/1936] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:03 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:03.302167 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6fa02-1e84-49e9-984f-701abc533198 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:03.313137 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6fa02-1e84-49e9-984f-701abc533198 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:03.320833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4f554-c8bb-49ea-a69e-d8be540185ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:03.321699 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4f554-c8bb-49ea-a69e-d8be540185ba admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:03.322004 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4f554-c8bb-49ea-a69e-d8be540185ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:03.322900 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4f554-c8bb-49ea-a69e-d8be540185ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:03.323580 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc4f554-c8bb-49ea-a69e-d8be540185ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:03.325239 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc4f554-c8bb-49ea-a69e-d8be540185ba admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:03.335646 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 968/1937] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:03.340010 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd0dc32-c752-479a-b998-85a6c5377a19 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:03.350377 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd0dc32-c752-479a-b998-85a6c5377a19 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:03.356165 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e730dc3-46b2-4882-8282-772526730310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:03.356779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e730dc3-46b2-4882-8282-772526730310 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:03.357061 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e730dc3-46b2-4882-8282-772526730310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:03.357335 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e730dc3-46b2-4882-8282-772526730310 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:03.357984 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e730dc3-46b2-4882-8282-772526730310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:03.359632 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e730dc3-46b2-4882-8282-772526730310 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:03.371688 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 970/1938] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:35:03.375996 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4f554-c8bb-49ea-a69e-d8be540185ba admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:03.386659 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4f554-c8bb-49ea-a69e-d8be540185ba admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:03.391277 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8408ff4-b4d7-4f45-8ee1-96241f4190c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:03.391939 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8408ff4-b4d7-4f45-8ee1-96241f4190c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:03.392221 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8408ff4-b4d7-4f45-8ee1-96241f4190c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:03.392496 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8408ff4-b4d7-4f45-8ee1-96241f4190c0 admin admin] PATH_INFO: `/v3/projects/9172bfdb59354caebe5ef1b7b48c629e/users/eaccdd8402ac4bdea26695b54a3b83a0/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:03.407664 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8408ff4-b4d7-4f45-8ee1-96241f4190c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9172bfdb59354caebe5ef1b7b48c629e, user_id=eaccdd8402ac4bdea26695b54a3b83a0, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:03.410432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8408ff4-b4d7-4f45-8ee1-96241f4190c0 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:03.424134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 969/1939] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:03 2022] PUT /identity/v3/projects/9172bfdb59354caebe5ef1b7b48c629e/users/eaccdd8402ac4bdea26695b54a3b83a0/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:03.434751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7f0bb-1eed-4c0f-83d9-ffeb0a8d304d None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:03.435037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7f0bb-1eed-4c0f-83d9-ffeb0a8d304d None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:03.435309 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7f0bb-1eed-4c0f-83d9-ffeb0a8d304d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:03.463830 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-05f7f0bb-1eed-4c0f-83d9-ffeb0a8d304d None None] MFA Rules not processed for user `eaccdd8402ac4bdea26695b54a3b83a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:03.465036 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-05f7f0bb-1eed-4c0f-83d9-ffeb0a8d304d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:03.473360 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:03.531815 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 971/1940] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:03.536340 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8408ff4-b4d7-4f45-8ee1-96241f4190c0 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:03.597953 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:03.616503 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8408ff4-b4d7-4f45-8ee1-96241f4190c0 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:03.624409 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c529a3-5649-4dbf-ad6d-5375213135a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:03.625131 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c529a3-5649-4dbf-ad6d-5375213135a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:03.625492 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c529a3-5649-4dbf-ad6d-5375213135a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:03.625838 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c529a3-5649-4dbf-ad6d-5375213135a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:03.626606 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c529a3-5649-4dbf-ad6d-5375213135a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:03.628701 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c529a3-5649-4dbf-ad6d-5375213135a9 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:03.661414 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 970/1941] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:03 2022] POST /identity/v3/projects => generated 387 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:03.665529 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7f0bb-1eed-4c0f-83d9-ffeb0a8d304d None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:03.679063 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7f0bb-1eed-4c0f-83d9-ffeb0a8d304d None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:03.742960 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42caf3d4-ae8a-42c0-99ca-9058bb1bf244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:03.743593 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42caf3d4-ae8a-42c0-99ca-9058bb1bf244 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:03.743903 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42caf3d4-ae8a-42c0-99ca-9058bb1bf244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:03.744866 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42caf3d4-ae8a-42c0-99ca-9058bb1bf244 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:03.745516 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42caf3d4-ae8a-42c0-99ca-9058bb1bf244 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:03.747590 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42caf3d4-ae8a-42c0-99ca-9058bb1bf244 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:03.780691 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 972/1942] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:03 2022] POST /identity/v3/users => generated 300 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:35:03.784913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c529a3-5649-4dbf-ad6d-5375213135a9 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:03.813339 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c529a3-5649-4dbf-ad6d-5375213135a9 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:03.818520 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75edfe80-3307-4295-b39f-b9cd80301b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:03.819034 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edfe80-3307-4295-b39f-b9cd80301b9c admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:03.827241 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edfe80-3307-4295-b39f-b9cd80301b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:03.827638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edfe80-3307-4295-b39f-b9cd80301b9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:03.828348 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75edfe80-3307-4295-b39f-b9cd80301b9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:03.830013 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75edfe80-3307-4295-b39f-b9cd80301b9c admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:03.846622 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 971/1943] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:03.851507 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42caf3d4-ae8a-42c0-99ca-9058bb1bf244 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:03.866666 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42caf3d4-ae8a-42c0-99ca-9058bb1bf244 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:03.874940 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f84d0b-07f3-47d8-aad8-dcd240b0263c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:03.876341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f84d0b-07f3-47d8-aad8-dcd240b0263c admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:03.876678 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f84d0b-07f3-47d8-aad8-dcd240b0263c admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:03.876891 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f84d0b-07f3-47d8-aad8-dcd240b0263c admin admin] PATH_INFO: `/v3/projects/da8dd601ff1149f1b577512d01256c10/users/2707a083757e4ea496b33272906191b4/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:03.901390 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f84d0b-07f3-47d8-aad8-dcd240b0263c admin admin] RBAC: Authorizing `identity:create_grant(project_id=da8dd601ff1149f1b577512d01256c10, user_id=2707a083757e4ea496b33272906191b4, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:03.904155 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f84d0b-07f3-47d8-aad8-dcd240b0263c admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:03.925521 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 973/1944] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:03 2022] PUT /identity/v3/projects/da8dd601ff1149f1b577512d01256c10/users/2707a083757e4ea496b33272906191b4/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:03.929228 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75edfe80-3307-4295-b39f-b9cd80301b9c admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:03.940458 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75edfe80-3307-4295-b39f-b9cd80301b9c admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:04.018858 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa8349-902e-4b72-8cdc-a7c6aa444b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:04.019295 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa8349-902e-4b72-8cdc-a7c6aa444b4f admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:04.019605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa8349-902e-4b72-8cdc-a7c6aa444b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:04.019879 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa8349-902e-4b72-8cdc-a7c6aa444b4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:04.020530 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa8349-902e-4b72-8cdc-a7c6aa444b4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:04.022130 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa8349-902e-4b72-8cdc-a7c6aa444b4f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:04.039035 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 972/1945] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:04.043930 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f84d0b-07f3-47d8-aad8-dcd240b0263c admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:04.056237 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f84d0b-07f3-47d8-aad8-dcd240b0263c admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:04.061627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c025b52a-01df-4a0c-89f9-905f1a8f60cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:04.062742 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025b52a-01df-4a0c-89f9-905f1a8f60cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:04.063410 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025b52a-01df-4a0c-89f9-905f1a8f60cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:04.063882 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025b52a-01df-4a0c-89f9-905f1a8f60cb admin admin] PATH_INFO: `/v3/projects/da8dd601ff1149f1b577512d01256c10/users/2707a083757e4ea496b33272906191b4/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:04.066531 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c025b52a-01df-4a0c-89f9-905f1a8f60cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=da8dd601ff1149f1b577512d01256c10, user_id=2707a083757e4ea496b33272906191b4, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:04.070544 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c025b52a-01df-4a0c-89f9-905f1a8f60cb admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:04.092326 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 974/1946] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:04 2022] PUT /identity/v3/projects/da8dd601ff1149f1b577512d01256c10/users/2707a083757e4ea496b33272906191b4/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:04.107500 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1afd52e-59f8-4208-813e-ffc64086e4ff None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:04.107792 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1afd52e-59f8-4208-813e-ffc64086e4ff None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:04.108071 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1afd52e-59f8-4208-813e-ffc64086e4ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:04.164555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-f1afd52e-59f8-4208-813e-ffc64086e4ff None None] MFA Rules not processed for user `2707a083757e4ea496b33272906191b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:04.169735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f1afd52e-59f8-4208-813e-ffc64086e4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:04.383523 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 973/1947] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:04 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:04.421186 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c025b52a-01df-4a0c-89f9-905f1a8f60cb admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:04.423634 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-c025b52a-01df-4a0c-89f9-905f1a8f60cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:04.528821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c025b52a-01df-4a0c-89f9-905f1a8f60cb admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:04.533277 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723b619-1414-4db1-bddc-39206cedf565 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:04.534390 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723b619-1414-4db1-bddc-39206cedf565 service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:04.534929 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723b619-1414-4db1-bddc-39206cedf565 service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:04.535442 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723b619-1414-4db1-bddc-39206cedf565 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:04.546657 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1723b619-1414-4db1-bddc-39206cedf565 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:04.548367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1723b619-1414-4db1-bddc-39206cedf565 service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:04.560656 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 975/1948] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:04 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:04.705374 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1afd52e-59f8-4208-813e-ffc64086e4ff None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:04.753112 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:04.772247 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1afd52e-59f8-4208-813e-ffc64086e4ff None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:04.778047 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58ee6d-0a4d-4c58-8f7d-64cdca512840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:04.778714 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58ee6d-0a4d-4c58-8f7d-64cdca512840 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:04.778992 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58ee6d-0a4d-4c58-8f7d-64cdca512840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:04.779270 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58ee6d-0a4d-4c58-8f7d-64cdca512840 admin admin] PATH_INFO: `/v3/users/eaccdd8402ac4bdea26695b54a3b83a0` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:04.780468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc58ee6d-0a4d-4c58-8f7d-64cdca512840 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaccdd8402ac4bdea26695b54a3b83a0)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:04.782405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc58ee6d-0a4d-4c58-8f7d-64cdca512840 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:04.828017 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-cc58ee6d-0a4d-4c58-8f7d-64cdca512840 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaccdd8402ac4bdea26695b54a3b83a0'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:04.834969 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-cc58ee6d-0a4d-4c58-8f7d-64cdca512840 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaccdd8402ac4bdea26695b54a3b83a0'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:04.835624 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-cc58ee6d-0a4d-4c58-8f7d-64cdca512840 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaccdd8402ac4bdea26695b54a3b83a0'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:04.847823 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-cc58ee6d-0a4d-4c58-8f7d-64cdca512840 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaccdd8402ac4bdea26695b54a3b83a0'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:04.848465 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-cc58ee6d-0a4d-4c58-8f7d-64cdca512840 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaccdd8402ac4bdea26695b54a3b83a0'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:04.862029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 974/1949] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:04 2022] DELETE /identity/v3/users/eaccdd8402ac4bdea26695b54a3b83a0 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:04.866544 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e891d-3339-467a-862f-a5eb238d2b9c None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:04.866829 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e891d-3339-467a-862f-a5eb238d2b9c None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:04.867099 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e891d-3339-467a-862f-a5eb238d2b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:04.876032 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:04.889887 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-089e891d-3339-467a-862f-a5eb238d2b9c None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:04.891152 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-089e891d-3339-467a-862f-a5eb238d2b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:04.959709 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 976/1950] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:04.970462 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58ee6d-0a4d-4c58-8f7d-64cdca512840 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:04.972299 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-cc58ee6d-0a4d-4c58-8f7d-64cdca512840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:05.032985 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58ee6d-0a4d-4c58-8f7d-64cdca512840 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:05.036916 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb2b7bd-c3e6-4a7b-b64f-3b1b80c23829 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:05.037532 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb2b7bd-c3e6-4a7b-b64f-3b1b80c23829 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:05.037816 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb2b7bd-c3e6-4a7b-b64f-3b1b80c23829 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:05.038093 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb2b7bd-c3e6-4a7b-b64f-3b1b80c23829 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:05.051137 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb2b7bd-c3e6-4a7b-b64f-3b1b80c23829 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:05.051345 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e891d-3339-467a-862f-a5eb238d2b9c None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:05.052737 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb2b7bd-c3e6-4a7b-b64f-3b1b80c23829 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:05.052897 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-089e891d-3339-467a-862f-a5eb238d2b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:05.064898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 975/1951] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:05.091979 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e891d-3339-467a-862f-a5eb238d2b9c None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:05.100566 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1055093b-4900-4803-8c5a-f6ec8ffa7d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:05.101735 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055093b-4900-4803-8c5a-f6ec8ffa7d9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:05.102481 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055093b-4900-4803-8c5a-f6ec8ffa7d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:05.103058 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055093b-4900-4803-8c5a-f6ec8ffa7d9b admin admin] PATH_INFO: `/v3/users/6aed24c9c7134e27bc3c516855ed0960` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:05.105531 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1055093b-4900-4803-8c5a-f6ec8ffa7d9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aed24c9c7134e27bc3c516855ed0960)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:05.109203 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1055093b-4900-4803-8c5a-f6ec8ffa7d9b admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:05.183825 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-1055093b-4900-4803-8c5a-f6ec8ffa7d9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aed24c9c7134e27bc3c516855ed0960'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:05.184737 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-1055093b-4900-4803-8c5a-f6ec8ffa7d9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aed24c9c7134e27bc3c516855ed0960'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:05.195844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-1055093b-4900-4803-8c5a-f6ec8ffa7d9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aed24c9c7134e27bc3c516855ed0960'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:05.196663 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-1055093b-4900-4803-8c5a-f6ec8ffa7d9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aed24c9c7134e27bc3c516855ed0960'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:05.221940 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-1055093b-4900-4803-8c5a-f6ec8ffa7d9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aed24c9c7134e27bc3c516855ed0960'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:05.239507 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 977/1952] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:05 2022] DELETE /identity/v3/users/6aed24c9c7134e27bc3c516855ed0960 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:05.244435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3fdca1-0432-4719-97e6-f666bbf38c87 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:05.244731 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3fdca1-0432-4719-97e6-f666bbf38c87 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:05.245013 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3fdca1-0432-4719-97e6-f666bbf38c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:05.268397 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-2f3fdca1-0432-4719-97e6-f666bbf38c87 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:05.269651 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2f3fdca1-0432-4719-97e6-f666bbf38c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:05.341241 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 976/1953] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:05.355505 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1055093b-4900-4803-8c5a-f6ec8ffa7d9b admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:05.357543 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-1055093b-4900-4803-8c5a-f6ec8ffa7d9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:05.444068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1055093b-4900-4803-8c5a-f6ec8ffa7d9b admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:05.450550 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c82fe-4f6c-4e3b-be17-fdd7ad4330d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:05.451457 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c82fe-4f6c-4e3b-be17-fdd7ad4330d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:05.451987 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c82fe-4f6c-4e3b-be17-fdd7ad4330d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:05.452286 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c82fe-4f6c-4e3b-be17-fdd7ad4330d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:05.466952 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c82fe-4f6c-4e3b-be17-fdd7ad4330d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:05.468844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c82fe-4f6c-4e3b-be17-fdd7ad4330d5 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:05.482915 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3fdca1-0432-4719-97e6-f666bbf38c87 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:05.485166 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2f3fdca1-0432-4719-97e6-f666bbf38c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:05.490960 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 978/1954] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:05.543094 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3fdca1-0432-4719-97e6-f666bbf38c87 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:05.564892 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84947b56-e6aa-423a-b61e-49c27c725592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:05.565560 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84947b56-e6aa-423a-b61e-49c27c725592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:05.565853 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84947b56-e6aa-423a-b61e-49c27c725592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:05.566484 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84947b56-e6aa-423a-b61e-49c27c725592 admin admin] PATH_INFO: `/v3/projects/9172bfdb59354caebe5ef1b7b48c629e` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:05.575711 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84947b56-e6aa-423a-b61e-49c27c725592 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9172bfdb59354caebe5ef1b7b48c629e)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:05.581206 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84947b56-e6aa-423a-b61e-49c27c725592 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:05.583439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:05.738806 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-84947b56-e6aa-423a-b61e-49c27c725592 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9172bfdb59354caebe5ef1b7b48c629e', 'request_id': 'req-84947b56-e6aa-423a-b61e-49c27c725592'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:05.750727 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-84947b56-e6aa-423a-b61e-49c27c725592 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9172bfdb59354caebe5ef1b7b48c629e', 'request_id': 'req-84947b56-e6aa-423a-b61e-49c27c725592'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:05.764169 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-84947b56-e6aa-423a-b61e-49c27c725592 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9172bfdb59354caebe5ef1b7b48c629e', 'request_id': 'req-84947b56-e6aa-423a-b61e-49c27c725592'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:05.784910 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-84947b56-e6aa-423a-b61e-49c27c725592 admin admin] The token cache is being invalidate because project 9172bfdb59354caebe5ef1b7b48c629e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:05.785368 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-84947b56-e6aa-423a-b61e-49c27c725592 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:05.788122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-84947b56-e6aa-423a-b61e-49c27c725592 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:05.793836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 977/1955] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:05 2022] DELETE /identity/v3/projects/9172bfdb59354caebe5ef1b7b48c629e => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:05.795426 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c82fe-4f6c-4e3b-be17-fdd7ad4330d5 service neutron] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:05.812975 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c82fe-4f6c-4e3b-be17-fdd7ad4330d5 service neutron] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:05.892093 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:05.958180 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db84ec8-c920-4768-8608-9dfdbb6dc8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:05.962973 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db84ec8-c920-4768-8608-9dfdbb6dc8b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:05.963731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db84ec8-c920-4768-8608-9dfdbb6dc8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:05.964625 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db84ec8-c920-4768-8608-9dfdbb6dc8b7 admin admin] PATH_INFO: `/v3/users/2707a083757e4ea496b33272906191b4` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:05.966705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db84ec8-c920-4768-8608-9dfdbb6dc8b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2707a083757e4ea496b33272906191b4)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:05.970439 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db84ec8-c920-4768-8608-9dfdbb6dc8b7 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:06.074470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-7db84ec8-c920-4768-8608-9dfdbb6dc8b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2707a083757e4ea496b33272906191b4'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:06.077145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-7db84ec8-c920-4768-8608-9dfdbb6dc8b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2707a083757e4ea496b33272906191b4'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:06.101609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-7db84ec8-c920-4768-8608-9dfdbb6dc8b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2707a083757e4ea496b33272906191b4'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:06.102459 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-7db84ec8-c920-4768-8608-9dfdbb6dc8b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2707a083757e4ea496b33272906191b4'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:06.126688 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-7db84ec8-c920-4768-8608-9dfdbb6dc8b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2707a083757e4ea496b33272906191b4'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:06.157699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 979/1956] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:05 2022] DELETE /identity/v3/users/2707a083757e4ea496b33272906191b4 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:06.466394 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84947b56-e6aa-423a-b61e-49c27c725592 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:06.467943 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-84947b56-e6aa-423a-b61e-49c27c725592 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:06.483689 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84947b56-e6aa-423a-b61e-49c27c725592 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:06.564155 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65098fbe-a037-4350-ab54-1ab517807a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:06.564916 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65098fbe-a037-4350-ab54-1ab517807a42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:06.565686 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65098fbe-a037-4350-ab54-1ab517807a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:06.565686 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65098fbe-a037-4350-ab54-1ab517807a42 admin admin] PATH_INFO: `/v3/projects/a141fb4b221348dab50db9f4ec785788` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:06.566788 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65098fbe-a037-4350-ab54-1ab517807a42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a141fb4b221348dab50db9f4ec785788)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:06.569016 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65098fbe-a037-4350-ab54-1ab517807a42 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:06.633216 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-65098fbe-a037-4350-ab54-1ab517807a42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a141fb4b221348dab50db9f4ec785788', 'request_id': 'req-65098fbe-a037-4350-ab54-1ab517807a42'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:06.640927 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-65098fbe-a037-4350-ab54-1ab517807a42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a141fb4b221348dab50db9f4ec785788', 'request_id': 'req-65098fbe-a037-4350-ab54-1ab517807a42'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:06.649366 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-65098fbe-a037-4350-ab54-1ab517807a42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a141fb4b221348dab50db9f4ec785788', 'request_id': 'req-65098fbe-a037-4350-ab54-1ab517807a42'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:06.668263 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-65098fbe-a037-4350-ab54-1ab517807a42 admin admin] The token cache is being invalidate because project a141fb4b221348dab50db9f4ec785788 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:06.669559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-65098fbe-a037-4350-ab54-1ab517807a42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:06.670093 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-65098fbe-a037-4350-ab54-1ab517807a42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:06.672169 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 978/1957] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:06 2022] DELETE /identity/v3/projects/a141fb4b221348dab50db9f4ec785788 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:06.675932 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db84ec8-c920-4768-8608-9dfdbb6dc8b7 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:06.677471 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-7db84ec8-c920-4768-8608-9dfdbb6dc8b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:06.692524 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db84ec8-c920-4768-8608-9dfdbb6dc8b7 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:06.751843 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65098fbe-a037-4350-ab54-1ab517807a42 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:06.753822 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-65098fbe-a037-4350-ab54-1ab517807a42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:06.775682 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e8b79-e191-441c-be93-7c4ea9cbd3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:06.776741 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e8b79-e191-441c-be93-7c4ea9cbd3b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:06.777339 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e8b79-e191-441c-be93-7c4ea9cbd3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:06.778059 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e8b79-e191-441c-be93-7c4ea9cbd3b0 admin admin] PATH_INFO: `/v3/users/4a798d5ed5cc458c8c88472bb10dc366` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:06.781827 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e8b79-e191-441c-be93-7c4ea9cbd3b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a798d5ed5cc458c8c88472bb10dc366)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:06.786934 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e8b79-e191-441c-be93-7c4ea9cbd3b0 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:06.804187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:06.851134 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:06.861787 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-314e8b79-e191-441c-be93-7c4ea9cbd3b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a798d5ed5cc458c8c88472bb10dc366'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:06.862632 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-314e8b79-e191-441c-be93-7c4ea9cbd3b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a798d5ed5cc458c8c88472bb10dc366'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:06.872388 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-314e8b79-e191-441c-be93-7c4ea9cbd3b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a798d5ed5cc458c8c88472bb10dc366'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:06.872939 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65098fbe-a037-4350-ab54-1ab517807a42 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:06.873104 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-314e8b79-e191-441c-be93-7c4ea9cbd3b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a798d5ed5cc458c8c88472bb10dc366'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:06.881764 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e93313-90c2-4dfe-9523-4ae8d06f7652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:06.882851 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e93313-90c2-4dfe-9523-4ae8d06f7652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:06.883315 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e93313-90c2-4dfe-9523-4ae8d06f7652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:06.885298 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e93313-90c2-4dfe-9523-4ae8d06f7652 admin admin] PATH_INFO: `/v3/projects/da8dd601ff1149f1b577512d01256c10` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:06.887297 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e93313-90c2-4dfe-9523-4ae8d06f7652 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da8dd601ff1149f1b577512d01256c10)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:06.888618 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-314e8b79-e191-441c-be93-7c4ea9cbd3b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a798d5ed5cc458c8c88472bb10dc366'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:06.889504 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e93313-90c2-4dfe-9523-4ae8d06f7652 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:06.907932 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 980/1958] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:06 2022] DELETE /identity/v3/users/4a798d5ed5cc458c8c88472bb10dc366 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:06.955780 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c2e93313-90c2-4dfe-9523-4ae8d06f7652 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da8dd601ff1149f1b577512d01256c10', 'request_id': 'req-c2e93313-90c2-4dfe-9523-4ae8d06f7652'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:06.964943 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c2e93313-90c2-4dfe-9523-4ae8d06f7652 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da8dd601ff1149f1b577512d01256c10', 'request_id': 'req-c2e93313-90c2-4dfe-9523-4ae8d06f7652'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:06.974478 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c2e93313-90c2-4dfe-9523-4ae8d06f7652 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da8dd601ff1149f1b577512d01256c10', 'request_id': 'req-c2e93313-90c2-4dfe-9523-4ae8d06f7652'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:06.988156 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c2e93313-90c2-4dfe-9523-4ae8d06f7652 admin admin] The token cache is being invalidate because project da8dd601ff1149f1b577512d01256c10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:06.988463 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c2e93313-90c2-4dfe-9523-4ae8d06f7652 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:06.989171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c2e93313-90c2-4dfe-9523-4ae8d06f7652 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:06.991746 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 979/1959] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:06 2022] DELETE /identity/v3/projects/da8dd601ff1149f1b577512d01256c10 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:06.998468 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4bc93-6639-42da-be3c-69c65724708c None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:06.998788 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4bc93-6639-42da-be3c-69c65724708c None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:06.999326 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4bc93-6639-42da-be3c-69c65724708c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:07.025716 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-96a4bc93-6639-42da-be3c-69c65724708c None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:07.027499 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-96a4bc93-6639-42da-be3c-69c65724708c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:07.138792 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 981/1960] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:07.158690 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c949a7c-c158-4e52-85b8-efba8a590113 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:07.159202 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c949a7c-c158-4e52-85b8-efba8a590113 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:07.159609 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c949a7c-c158-4e52-85b8-efba8a590113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:07.190894 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-6c949a7c-c158-4e52-85b8-efba8a590113 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:07.192140 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6c949a7c-c158-4e52-85b8-efba8a590113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:07.201381 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 980/1961] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:07.205632 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4bc93-6639-42da-be3c-69c65724708c None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:07.233100 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4bc93-6639-42da-be3c-69c65724708c None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:07.251916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f4ec6-81da-425f-a339-10b93542c1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:07.252571 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f4ec6-81da-425f-a339-10b93542c1c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:07.252976 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f4ec6-81da-425f-a339-10b93542c1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:07.253603 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f4ec6-81da-425f-a339-10b93542c1c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:07.253984 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2f4ec6-81da-425f-a339-10b93542c1c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:35:07.254270 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2f4ec6-81da-425f-a339-10b93542c1c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:07.256159 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2f4ec6-81da-425f-a339-10b93542c1c2 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:07.284128 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 982/1962] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:35:07.285430 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c949a7c-c158-4e52-85b8-efba8a590113 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:07.296165 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c949a7c-c158-4e52-85b8-efba8a590113 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:07.302115 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad2508-1b63-4155-9fb8-f478cbf642af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:07.304011 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad2508-1b63-4155-9fb8-f478cbf642af admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:07.304510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad2508-1b63-4155-9fb8-f478cbf642af admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:07.305066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad2508-1b63-4155-9fb8-f478cbf642af admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:07.305767 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad2508-1b63-4155-9fb8-f478cbf642af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:07.307828 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad2508-1b63-4155-9fb8-f478cbf642af admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:07.355449 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f4ec6-81da-425f-a339-10b93542c1c2 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:07.367483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 981/1963] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:07 2022] POST /identity/v3/projects => generated 385 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:07.376773 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f4ec6-81da-425f-a339-10b93542c1c2 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:07.469476 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53adf42-d505-4357-9441-9a69c05059f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:07.470617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53adf42-d505-4357-9441-9a69c05059f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:07.471144 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53adf42-d505-4357-9441-9a69c05059f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:07.471655 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53adf42-d505-4357-9441-9a69c05059f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:07.472610 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53adf42-d505-4357-9441-9a69c05059f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:07.474931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53adf42-d505-4357-9441-9a69c05059f0 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:07.533863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 983/1964] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:07 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:35:07.538199 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad2508-1b63-4155-9fb8-f478cbf642af admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:07.562424 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad2508-1b63-4155-9fb8-f478cbf642af admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:07.568870 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891fbb1-8229-41ee-99da-ce7feb04fdd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:07.570632 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891fbb1-8229-41ee-99da-ce7feb04fdd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:07.571179 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891fbb1-8229-41ee-99da-ce7feb04fdd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:07.571752 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891fbb1-8229-41ee-99da-ce7feb04fdd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:07.573117 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d891fbb1-8229-41ee-99da-ce7feb04fdd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:07.575346 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d891fbb1-8229-41ee-99da-ce7feb04fdd1 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:07.587766 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 982/1965] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:07.591865 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53adf42-d505-4357-9441-9a69c05059f0 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:07.617543 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53adf42-d505-4357-9441-9a69c05059f0 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:07.644450 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b4c9a-8d37-46cd-b363-b87e8a316206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:07.644450 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b4c9a-8d37-46cd-b363-b87e8a316206 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:07.644450 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b4c9a-8d37-46cd-b363-b87e8a316206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:07.644450 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b4c9a-8d37-46cd-b363-b87e8a316206 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:07.645178 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0b4c9a-8d37-46cd-b363-b87e8a316206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:07.647017 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0b4c9a-8d37-46cd-b363-b87e8a316206 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:07.673670 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 984/1966] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:07.685223 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891fbb1-8229-41ee-99da-ce7feb04fdd1 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:07.695692 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891fbb1-8229-41ee-99da-ce7feb04fdd1 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:07.700148 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b4c9a-8d37-46cd-b363-b87e8a316206 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:07.707079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-1f0b4c9a-8d37-46cd-b363-b87e8a316206 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:07.717901 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b50693-482e-419a-b9ad-8c50734f259e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:07.718817 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b50693-482e-419a-b9ad-8c50734f259e admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:07.719597 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b50693-482e-419a-b9ad-8c50734f259e admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:07.723936 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b50693-482e-419a-b9ad-8c50734f259e admin admin] PATH_INFO: `/v3/projects/a518ccfe8a1644599bcbcadbc57be421/users/9e95a3af2acf406d9976e2f9bd24cec2/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:07.752980 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b4c9a-8d37-46cd-b363-b87e8a316206 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:07.765060 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756143e1-cd37-4edc-ab42-1308e690ead6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:07.766344 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756143e1-cd37-4edc-ab42-1308e690ead6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:07.766344 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756143e1-cd37-4edc-ab42-1308e690ead6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:07.766506 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756143e1-cd37-4edc-ab42-1308e690ead6 admin admin] PATH_INFO: `/v3/projects/40b47ea752e14a1c97fa0a540dbac92f` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:07.767836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b50693-482e-419a-b9ad-8c50734f259e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a518ccfe8a1644599bcbcadbc57be421, user_id=9e95a3af2acf406d9976e2f9bd24cec2, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:07.768292 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756143e1-cd37-4edc-ab42-1308e690ead6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40b47ea752e14a1c97fa0a540dbac92f)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:07.771281 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756143e1-cd37-4edc-ab42-1308e690ead6 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:07.780734 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b50693-482e-419a-b9ad-8c50734f259e admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:07.801567 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 983/1967] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:07 2022] PUT /identity/v3/projects/a518ccfe8a1644599bcbcadbc57be421/users/9e95a3af2acf406d9976e2f9bd24cec2/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:07.813204 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10fa6e9-d72d-48ab-9758-a9cdb198c3f1 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:07.813496 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10fa6e9-d72d-48ab-9758-a9cdb198c3f1 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:07.814453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10fa6e9-d72d-48ab-9758-a9cdb198c3f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:07.838568 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:07.857377 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-f10fa6e9-d72d-48ab-9758-a9cdb198c3f1 None None] MFA Rules not processed for user `9e95a3af2acf406d9976e2f9bd24cec2`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:07.859008 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f10fa6e9-d72d-48ab-9758-a9cdb198c3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:07.890841 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-756143e1-cd37-4edc-ab42-1308e690ead6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40b47ea752e14a1c97fa0a540dbac92f', 'request_id': 'req-756143e1-cd37-4edc-ab42-1308e690ead6'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:07.904014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-756143e1-cd37-4edc-ab42-1308e690ead6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40b47ea752e14a1c97fa0a540dbac92f', 'request_id': 'req-756143e1-cd37-4edc-ab42-1308e690ead6'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:07.918552 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-756143e1-cd37-4edc-ab42-1308e690ead6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40b47ea752e14a1c97fa0a540dbac92f', 'request_id': 'req-756143e1-cd37-4edc-ab42-1308e690ead6'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:07.918705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:07.932505 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-756143e1-cd37-4edc-ab42-1308e690ead6 admin admin] The token cache is being invalidate because project 40b47ea752e14a1c97fa0a540dbac92f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:07.932894 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-756143e1-cd37-4edc-ab42-1308e690ead6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:07.933688 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-756143e1-cd37-4edc-ab42-1308e690ead6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:07.936118 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 985/1968] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:07 2022] DELETE /identity/v3/projects/40b47ea752e14a1c97fa0a540dbac92f => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:07.943404 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756143e1-cd37-4edc-ab42-1308e690ead6 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:07.946759 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-756143e1-cd37-4edc-ab42-1308e690ead6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:07.968736 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 984/1969] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:35:07.969194 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb561c-c56c-4070-bfaf-5826a9ef9d9f None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:07.969738 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb561c-c56c-4070-bfaf-5826a9ef9d9f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:07.970146 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb561c-c56c-4070-bfaf-5826a9ef9d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:08.006516 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-33fb561c-c56c-4070-bfaf-5826a9ef9d9f None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:08.008191 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-33fb561c-c56c-4070-bfaf-5826a9ef9d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:08.046996 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756143e1-cd37-4edc-ab42-1308e690ead6 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:08.053116 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3fabd-bf1a-427f-978e-5a0b541b9c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:08.053790 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3fabd-bf1a-427f-978e-5a0b541b9c68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:08.054077 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3fabd-bf1a-427f-978e-5a0b541b9c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:08.054395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3fabd-bf1a-427f-978e-5a0b541b9c68 admin admin] PATH_INFO: `/v3/users/3e47f24f10de4f8da90de9cf111106eb` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:08.055632 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf3fabd-bf1a-427f-978e-5a0b541b9c68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e47f24f10de4f8da90de9cf111106eb)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:08.058200 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf3fabd-bf1a-427f-978e-5a0b541b9c68 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:08.085714 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 985/1970] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:08.087067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb561c-c56c-4070-bfaf-5826a9ef9d9f None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:08.088120 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-33fb561c-c56c-4070-bfaf-5826a9ef9d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:08.117987 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-baf3fabd-bf1a-427f-978e-5a0b541b9c68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e47f24f10de4f8da90de9cf111106eb'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:08.119543 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-baf3fabd-bf1a-427f-978e-5a0b541b9c68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e47f24f10de4f8da90de9cf111106eb'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:08.129640 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-baf3fabd-bf1a-427f-978e-5a0b541b9c68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e47f24f10de4f8da90de9cf111106eb'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:08.134414 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-baf3fabd-bf1a-427f-978e-5a0b541b9c68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e47f24f10de4f8da90de9cf111106eb'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:08.154057 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb561c-c56c-4070-bfaf-5826a9ef9d9f None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:08.158012 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba68e36-01e8-4c8c-808c-965000cce9ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:08.158733 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba68e36-01e8-4c8c-808c-965000cce9ac service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:08.159016 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba68e36-01e8-4c8c-808c-965000cce9ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:08.159318 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba68e36-01e8-4c8c-808c-965000cce9ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:08.161147 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-dba68e36-01e8-4c8c-808c-965000cce9ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:08.163664 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-baf3fabd-bf1a-427f-978e-5a0b541b9c68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e47f24f10de4f8da90de9cf111106eb'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:08.176428 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba68e36-01e8-4c8c-808c-965000cce9ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:08.177892 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba68e36-01e8-4c8c-808c-965000cce9ac service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:08.185469 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 986/1971] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:07 2022] DELETE /identity/v3/users/3e47f24f10de4f8da90de9cf111106eb => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:08.187100 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b011c6-05ff-47f2-a413-848dc2909d2a None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:08.187476 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b011c6-05ff-47f2-a413-848dc2909d2a None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:08.188175 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b011c6-05ff-47f2-a413-848dc2909d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:08.189128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 986/1972] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:08.190442 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6a3b3-578b-4dbb-a2c6-c909605b22e4 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:08.190704 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6a3b3-578b-4dbb-a2c6-c909605b22e4 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:08.191014 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6a3b3-578b-4dbb-a2c6-c909605b22e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:08.210697 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-1fd6a3b3-578b-4dbb-a2c6-c909605b22e4 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:08.212271 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-1fd6a3b3-578b-4dbb-a2c6-c909605b22e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:08.233221 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-b0b011c6-05ff-47f2-a413-848dc2909d2a None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:08.236563 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-b0b011c6-05ff-47f2-a413-848dc2909d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:08.265175 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:08.317573 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 987/1973] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:08.319740 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd6a3b3-578b-4dbb-a2c6-c909605b22e4 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:08.321227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-1fd6a3b3-578b-4dbb-a2c6-c909605b22e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:08.339543 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd6a3b3-578b-4dbb-a2c6-c909605b22e4 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:08.340524 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 987/1974] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:08.341581 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b011c6-05ff-47f2-a413-848dc2909d2a None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:08.343035 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-b0b011c6-05ff-47f2-a413-848dc2909d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:08.355403 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a088d4c-e378-400a-83b7-16216dab93cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:08.356672 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a088d4c-e378-400a-83b7-16216dab93cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:08.356962 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a088d4c-e378-400a-83b7-16216dab93cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:08.357246 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a088d4c-e378-400a-83b7-16216dab93cb admin admin] PATH_INFO: `/v3/users/9e95a3af2acf406d9976e2f9bd24cec2` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:08.361645 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a088d4c-e378-400a-83b7-16216dab93cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e95a3af2acf406d9976e2f9bd24cec2)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:08.370475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a088d4c-e378-400a-83b7-16216dab93cb admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:08.400548 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:08.434038 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b011c6-05ff-47f2-a413-848dc2909d2a None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:08.439048 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95aa5ec-c2f5-4740-8ce4-b3b0f3b07d9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:08.439840 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95aa5ec-c2f5-4740-8ce4-b3b0f3b07d9e service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:08.440294 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95aa5ec-c2f5-4740-8ce4-b3b0f3b07d9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:08.440722 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95aa5ec-c2f5-4740-8ce4-b3b0f3b07d9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:08.453090 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95aa5ec-c2f5-4740-8ce4-b3b0f3b07d9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:08.455038 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95aa5ec-c2f5-4740-8ce4-b3b0f3b07d9e service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:08.480277 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9a088d4c-e378-400a-83b7-16216dab93cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e95a3af2acf406d9976e2f9bd24cec2'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:08.486534 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 988/1975] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:08.489637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95aa5ec-c2f5-4740-8ce4-b3b0f3b07d9e service neutron] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:08.501098 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95aa5ec-c2f5-4740-8ce4-b3b0f3b07d9e service neutron] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:08.506322 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9a088d4c-e378-400a-83b7-16216dab93cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e95a3af2acf406d9976e2f9bd24cec2'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:08.507445 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9a088d4c-e378-400a-83b7-16216dab93cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e95a3af2acf406d9976e2f9bd24cec2'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:08.507575 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c46b759-8467-4823-a52f-32c42f49a66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:08.508133 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46b759-8467-4823-a52f-32c42f49a66b admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:08.508422 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46b759-8467-4823-a52f-32c42f49a66b admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:08.508699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46b759-8467-4823-a52f-32c42f49a66b admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:08.509384 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c46b759-8467-4823-a52f-32c42f49a66b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:35:08.509696 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c46b759-8467-4823-a52f-32c42f49a66b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:08.511518 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c46b759-8467-4823-a52f-32c42f49a66b admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:08.524391 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 989/1976] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:35:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:35:08.529517 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c46b759-8467-4823-a52f-32c42f49a66b admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:08.540190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9a088d4c-e378-400a-83b7-16216dab93cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e95a3af2acf406d9976e2f9bd24cec2'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:08.541529 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9a088d4c-e378-400a-83b7-16216dab93cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e95a3af2acf406d9976e2f9bd24cec2'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:08.545958 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c46b759-8467-4823-a52f-32c42f49a66b admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:08.553269 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551bd90e-6288-456d-aca1-da7b73946769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:08.554337 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bd90e-6288-456d-aca1-da7b73946769 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:08.554731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bd90e-6288-456d-aca1-da7b73946769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:08.555160 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bd90e-6288-456d-aca1-da7b73946769 admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:08.555921 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551bd90e-6288-456d-aca1-da7b73946769 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:08.556289 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 988/1977] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:08 2022] DELETE /identity/v3/users/9e95a3af2acf406d9976e2f9bd24cec2 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:08.561712 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa51dc-d01b-4349-90db-d38d8e9f5980 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:08.562307 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa51dc-d01b-4349-90db-d38d8e9f5980 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:08.563097 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa51dc-d01b-4349-90db-d38d8e9f5980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:08.566169 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551bd90e-6288-456d-aca1-da7b73946769 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:08.585024 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-8dfa51dc-d01b-4349-90db-d38d8e9f5980 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:08.586689 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8dfa51dc-d01b-4349-90db-d38d8e9f5980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:08.604506 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 990/1978] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:08 2022] POST /identity/v3/projects => generated 373 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:35:08.608079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551bd90e-6288-456d-aca1-da7b73946769 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:08.615037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-551bd90e-6288-456d-aca1-da7b73946769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:08.658794 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 989/1979] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:35:08.669818 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa51dc-d01b-4349-90db-d38d8e9f5980 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:08.671675 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8dfa51dc-d01b-4349-90db-d38d8e9f5980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:08.715471 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551bd90e-6288-456d-aca1-da7b73946769 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:08.725260 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592a2ea-c8e7-4d16-ac3f-aeb7c38fda40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:08.726459 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592a2ea-c8e7-4d16-ac3f-aeb7c38fda40 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:08.726751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592a2ea-c8e7-4d16-ac3f-aeb7c38fda40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:08.727034 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592a2ea-c8e7-4d16-ac3f-aeb7c38fda40 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:08.727691 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2592a2ea-c8e7-4d16-ac3f-aeb7c38fda40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:08.730424 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2592a2ea-c8e7-4d16-ac3f-aeb7c38fda40 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:08.763725 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 991/1980] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:08 2022] POST /identity/v3/users => generated 287 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:35:08.765804 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa51dc-d01b-4349-90db-d38d8e9f5980 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:08.767324 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592a2ea-c8e7-4d16-ac3f-aeb7c38fda40 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:08.781548 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592a2ea-c8e7-4d16-ac3f-aeb7c38fda40 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:08.788445 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22692e4-596b-4091-9cee-43b0da2cec81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:08.789629 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22692e4-596b-4091-9cee-43b0da2cec81 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:08.789918 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22692e4-596b-4091-9cee-43b0da2cec81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:08.791344 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22692e4-596b-4091-9cee-43b0da2cec81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:08.792075 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22692e4-596b-4091-9cee-43b0da2cec81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:08.796120 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22692e4-596b-4091-9cee-43b0da2cec81 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:08.811812 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 992/1981] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:08.815629 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22692e4-596b-4091-9cee-43b0da2cec81 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:08.829085 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22692e4-596b-4091-9cee-43b0da2cec81 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:08.837649 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb1855c-f7a6-430f-8eeb-e4fb84162243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:08.838456 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1855c-f7a6-430f-8eeb-e4fb84162243 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:08.839325 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1855c-f7a6-430f-8eeb-e4fb84162243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:08.839853 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1855c-f7a6-430f-8eeb-e4fb84162243 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:08.840605 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb1855c-f7a6-430f-8eeb-e4fb84162243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:08.842980 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb1855c-f7a6-430f-8eeb-e4fb84162243 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:08.859568 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 993/1982] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:08.861816 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:08.863867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb1855c-f7a6-430f-8eeb-e4fb84162243 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:08.872414 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:08.877592 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3eeefd-0053-4c36-b1f2-e97be2282bd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:08.879999 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3eeefd-0053-4c36-b1f2-e97be2282bd5 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:08.880294 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3eeefd-0053-4c36-b1f2-e97be2282bd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:08.880586 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3eeefd-0053-4c36-b1f2-e97be2282bd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:08.881634 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb1855c-f7a6-430f-8eeb-e4fb84162243 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:08.888559 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51104c-feb0-4cfe-95c3-0e0c98335cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:08.890378 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51104c-feb0-4cfe-95c3-0e0c98335cce admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:08.891035 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51104c-feb0-4cfe-95c3-0e0c98335cce admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:08.891524 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51104c-feb0-4cfe-95c3-0e0c98335cce admin admin] PATH_INFO: `/v3/projects/0cc4a244d7674e6196305091482c90b5/users/130f67aab4344f7caaac517374e5dabd/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:08.903951 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3eeefd-0053-4c36-b1f2-e97be2282bd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:08.908122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3eeefd-0053-4c36-b1f2-e97be2282bd5 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:08.913804 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e51104c-feb0-4cfe-95c3-0e0c98335cce admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cc4a244d7674e6196305091482c90b5, user_id=130f67aab4344f7caaac517374e5dabd, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:08.918814 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e51104c-feb0-4cfe-95c3-0e0c98335cce admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:08.927148 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 990/1983] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:08.945966 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 994/1984] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:08 2022] PUT /identity/v3/projects/0cc4a244d7674e6196305091482c90b5/users/130f67aab4344f7caaac517374e5dabd/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:08.951851 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236e413-3d7f-4d31-ac31-fb92504f61d6 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:08.952288 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236e413-3d7f-4d31-ac31-fb92504f61d6 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:08.952646 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236e413-3d7f-4d31-ac31-fb92504f61d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:08.999997 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-7236e413-3d7f-4d31-ac31-fb92504f61d6 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:09.003106 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7236e413-3d7f-4d31-ac31-fb92504f61d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:09.170400 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 991/1985] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:09.195991 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51104c-feb0-4cfe-95c3-0e0c98335cce admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:09.309132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51104c-feb0-4cfe-95c3-0e0c98335cce admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:09.315723 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba00d81-a4cf-46bc-b2a0-bb0109192789 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:09.318717 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba00d81-a4cf-46bc-b2a0-bb0109192789 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:09.320051 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba00d81-a4cf-46bc-b2a0-bb0109192789 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:09.320796 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba00d81-a4cf-46bc-b2a0-bb0109192789 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:09.340611 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba00d81-a4cf-46bc-b2a0-bb0109192789 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:09.343049 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba00d81-a4cf-46bc-b2a0-bb0109192789 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:09.372434 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 995/1986] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:09.433848 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236e413-3d7f-4d31-ac31-fb92504f61d6 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:09.435962 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7236e413-3d7f-4d31-ac31-fb92504f61d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:09.454151 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236e413-3d7f-4d31-ac31-fb92504f61d6 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:09.460432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebab2d7-ac70-4d6d-84b0-622e8f666e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:09.461202 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebab2d7-ac70-4d6d-84b0-622e8f666e0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:09.462222 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebab2d7-ac70-4d6d-84b0-622e8f666e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:09.462786 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebab2d7-ac70-4d6d-84b0-622e8f666e0b admin admin] PATH_INFO: `/v3/projects/a518ccfe8a1644599bcbcadbc57be421` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:09.464570 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebab2d7-ac70-4d6d-84b0-622e8f666e0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a518ccfe8a1644599bcbcadbc57be421)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:09.468082 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebab2d7-ac70-4d6d-84b0-622e8f666e0b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:09.574126 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-bebab2d7-ac70-4d6d-84b0-622e8f666e0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a518ccfe8a1644599bcbcadbc57be421', 'request_id': 'req-bebab2d7-ac70-4d6d-84b0-622e8f666e0b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:09.583477 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-bebab2d7-ac70-4d6d-84b0-622e8f666e0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a518ccfe8a1644599bcbcadbc57be421', 'request_id': 'req-bebab2d7-ac70-4d6d-84b0-622e8f666e0b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:09.601618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-bebab2d7-ac70-4d6d-84b0-622e8f666e0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a518ccfe8a1644599bcbcadbc57be421', 'request_id': 'req-bebab2d7-ac70-4d6d-84b0-622e8f666e0b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:09.620167 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-bebab2d7-ac70-4d6d-84b0-622e8f666e0b admin admin] The token cache is being invalidate because project a518ccfe8a1644599bcbcadbc57be421 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:09.620543 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-bebab2d7-ac70-4d6d-84b0-622e8f666e0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:09.622920 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-bebab2d7-ac70-4d6d-84b0-622e8f666e0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:09.624828 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 992/1987] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:09 2022] DELETE /identity/v3/projects/a518ccfe8a1644599bcbcadbc57be421 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:09.630512 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7223e7e-681c-48e1-9c14-f9b359bbf797 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:09.630890 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7223e7e-681c-48e1-9c14-f9b359bbf797 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:09.631244 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7223e7e-681c-48e1-9c14-f9b359bbf797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:09.662113 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-a7223e7e-681c-48e1-9c14-f9b359bbf797 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:09.664923 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-a7223e7e-681c-48e1-9c14-f9b359bbf797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:09.750294 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 996/1988] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:09.758584 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a1fe7-8ce4-4bee-8750-e879d381a619 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:09.758584 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a1fe7-8ce4-4bee-8750-e879d381a619 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:09.758584 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a1fe7-8ce4-4bee-8750-e879d381a619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:09.779865 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-ef4a1fe7-8ce4-4bee-8750-e879d381a619 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:09.780913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ef4a1fe7-8ce4-4bee-8750-e879d381a619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:09.787467 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7223e7e-681c-48e1-9c14-f9b359bbf797 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:09.788881 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-a7223e7e-681c-48e1-9c14-f9b359bbf797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:09.803128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 993/1989] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:35:09.803346 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7223e7e-681c-48e1-9c14-f9b359bbf797 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:09.804300 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4a1fe7-8ce4-4bee-8750-e879d381a619 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:09.808049 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f42d9b-e326-419a-9340-9c4576002c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:09.808702 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f42d9b-e326-419a-9340-9c4576002c63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:09.808984 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f42d9b-e326-419a-9340-9c4576002c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:09.809257 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f42d9b-e326-419a-9340-9c4576002c63 admin admin] PATH_INFO: `/v3/projects/4a2d39ce733640059b3d367e21ba9caa` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:09.810725 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f42d9b-e326-419a-9340-9c4576002c63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a2d39ce733640059b3d367e21ba9caa)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:09.813217 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f42d9b-e326-419a-9340-9c4576002c63 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:09.814692 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4a1fe7-8ce4-4bee-8750-e879d381a619 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:09.819869 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d601f3a-43a2-4a8b-85b8-ef93e6df0e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:09.820923 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d601f3a-43a2-4a8b-85b8-ef93e6df0e2b admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:09.821436 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d601f3a-43a2-4a8b-85b8-ef93e6df0e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:09.821949 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d601f3a-43a2-4a8b-85b8-ef93e6df0e2b admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:09.822837 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d601f3a-43a2-4a8b-85b8-ef93e6df0e2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:35:09.823347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d601f3a-43a2-4a8b-85b8-ef93e6df0e2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:09.825160 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d601f3a-43a2-4a8b-85b8-ef93e6df0e2b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:09.841888 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 994/1990] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:35:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:35:09.846404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d601f3a-43a2-4a8b-85b8-ef93e6df0e2b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:09.867597 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d601f3a-43a2-4a8b-85b8-ef93e6df0e2b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:09.879750 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-b3f42d9b-e326-419a-9340-9c4576002c63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a2d39ce733640059b3d367e21ba9caa', 'request_id': 'req-b3f42d9b-e326-419a-9340-9c4576002c63'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:09.899226 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-b3f42d9b-e326-419a-9340-9c4576002c63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a2d39ce733640059b3d367e21ba9caa', 'request_id': 'req-b3f42d9b-e326-419a-9340-9c4576002c63'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:09.908486 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-b3f42d9b-e326-419a-9340-9c4576002c63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a2d39ce733640059b3d367e21ba9caa', 'request_id': 'req-b3f42d9b-e326-419a-9340-9c4576002c63'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:09.925509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-b3f42d9b-e326-419a-9340-9c4576002c63 admin admin] The token cache is being invalidate because project 4a2d39ce733640059b3d367e21ba9caa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:09.925814 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-b3f42d9b-e326-419a-9340-9c4576002c63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:09.926657 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-b3f42d9b-e326-419a-9340-9c4576002c63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:09.928556 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 997/1991] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:09 2022] DELETE /identity/v3/projects/4a2d39ce733640059b3d367e21ba9caa => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:09.933995 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f42d9b-e326-419a-9340-9c4576002c63 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:09.936021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-b3f42d9b-e326-419a-9340-9c4576002c63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:09.944731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:09.966870 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:09.979061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba170c31-f858-48c1-ab56-2bc782598fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:09.980705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba170c31-f858-48c1-ab56-2bc782598fa6 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:09.981347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba170c31-f858-48c1-ab56-2bc782598fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:09.981885 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba170c31-f858-48c1-ab56-2bc782598fa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:09.983699 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba170c31-f858-48c1-ab56-2bc782598fa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:09.986388 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba170c31-f858-48c1-ab56-2bc782598fa6 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:10.037065 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 995/1992] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:09 2022] POST /identity/v3/projects => generated 369 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:10.041630 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f42d9b-e326-419a-9340-9c4576002c63 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:10.043068 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba170c31-f858-48c1-ab56-2bc782598fa6 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:10.049311 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ba170c31-f858-48c1-ab56-2bc782598fa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:10.120990 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dcf34f-8c1e-4413-ac3b-75a8dc947abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:10.121740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dcf34f-8c1e-4413-ac3b-75a8dc947abc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:10.122122 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dcf34f-8c1e-4413-ac3b-75a8dc947abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:10.122530 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dcf34f-8c1e-4413-ac3b-75a8dc947abc admin admin] PATH_INFO: `/v3/users/2142cb3fe0b9436997488d840c46ccad` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:10.123926 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dcf34f-8c1e-4413-ac3b-75a8dc947abc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2142cb3fe0b9436997488d840c46ccad)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:10.126428 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dcf34f-8c1e-4413-ac3b-75a8dc947abc admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:10.139258 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba170c31-f858-48c1-ab56-2bc782598fa6 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:10.146664 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead3ba6-4aac-445d-9090-3339c31749ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:10.147950 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead3ba6-4aac-445d-9090-3339c31749ba admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:10.148380 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead3ba6-4aac-445d-9090-3339c31749ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:10.148967 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead3ba6-4aac-445d-9090-3339c31749ba admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:10.150636 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dead3ba6-4aac-445d-9090-3339c31749ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:10.160035 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dead3ba6-4aac-445d-9090-3339c31749ba admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:10.189353 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 996/1993] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:10 2022] POST /identity/v3/users => generated 285 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:10.193440 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead3ba6-4aac-445d-9090-3339c31749ba admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:10.204082 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead3ba6-4aac-445d-9090-3339c31749ba admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:10.204283 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-e7dcf34f-8c1e-4413-ac3b-75a8dc947abc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2142cb3fe0b9436997488d840c46ccad'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:10.205379 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-e7dcf34f-8c1e-4413-ac3b-75a8dc947abc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2142cb3fe0b9436997488d840c46ccad'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:10.209051 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253863fb-f908-47b1-8b5f-a8c4595f3e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:10.209856 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253863fb-f908-47b1-8b5f-a8c4595f3e5a admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:10.210007 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253863fb-f908-47b1-8b5f-a8c4595f3e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:10.210403 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253863fb-f908-47b1-8b5f-a8c4595f3e5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:10.211046 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253863fb-f908-47b1-8b5f-a8c4595f3e5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:10.212743 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253863fb-f908-47b1-8b5f-a8c4595f3e5a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:10.215986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-e7dcf34f-8c1e-4413-ac3b-75a8dc947abc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2142cb3fe0b9436997488d840c46ccad'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:10.217645 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-e7dcf34f-8c1e-4413-ac3b-75a8dc947abc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2142cb3fe0b9436997488d840c46ccad'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:10.226859 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 997/1994] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:35:10.229596 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253863fb-f908-47b1-8b5f-a8c4595f3e5a admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:10.231327 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-253863fb-f908-47b1-8b5f-a8c4595f3e5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:10.235269 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-e7dcf34f-8c1e-4413-ac3b-75a8dc947abc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2142cb3fe0b9436997488d840c46ccad'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:10.245167 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253863fb-f908-47b1-8b5f-a8c4595f3e5a admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:10.250104 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf5093-9c56-44d5-9955-f883d8311cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:10.250812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf5093-9c56-44d5-9955-f883d8311cad admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:10.251125 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf5093-9c56-44d5-9955-f883d8311cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:10.251403 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf5093-9c56-44d5-9955-f883d8311cad admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:10.252079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf5093-9c56-44d5-9955-f883d8311cad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:10.253543 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 998/1995] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:09 2022] DELETE /identity/v3/users/2142cb3fe0b9436997488d840c46ccad => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:10.253663 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf5093-9c56-44d5-9955-f883d8311cad admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:10.274285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 998/1996] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:10 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:35:10.279352 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dcf34f-8c1e-4413-ac3b-75a8dc947abc admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:10.320302 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:10.362214 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dcf34f-8c1e-4413-ac3b-75a8dc947abc admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:10.372141 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b83754-6104-40c2-a51f-a0ce0ec02f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:10.373245 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b83754-6104-40c2-a51f-a0ce0ec02f18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:10.373245 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b83754-6104-40c2-a51f-a0ce0ec02f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:10.373553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b83754-6104-40c2-a51f-a0ce0ec02f18 admin admin] PATH_INFO: `/v3/projects/0939af057f4940298089c8f89ec2865f/users/7425d9aefcf940709f3a3aad4166755c/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:10.392127 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b83754-6104-40c2-a51f-a0ce0ec02f18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0939af057f4940298089c8f89ec2865f, user_id=7425d9aefcf940709f3a3aad4166755c, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:10.395170 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b83754-6104-40c2-a51f-a0ce0ec02f18 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:10.413119 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 999/1997] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:10 2022] PUT /identity/v3/projects/0939af057f4940298089c8f89ec2865f/users/7425d9aefcf940709f3a3aad4166755c/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:10.419471 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab54b3-1ec8-4822-9505-15f809a3d48d None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:10.419759 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab54b3-1ec8-4822-9505-15f809a3d48d None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:10.420038 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab54b3-1ec8-4822-9505-15f809a3d48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:10.433757 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb9cfa-fba9-4a81-9a6b-48eb44adffd0 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:10.434713 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb9cfa-fba9-4a81-9a6b-48eb44adffd0 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:10.435164 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb9cfa-fba9-4a81-9a6b-48eb44adffd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:10.449469 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-21ab54b3-1ec8-4822-9505-15f809a3d48d None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:10.450833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-21ab54b3-1ec8-4822-9505-15f809a3d48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:10.476031 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-8eeb9cfa-fba9-4a81-9a6b-48eb44adffd0 None None] MFA Rules not processed for user `130f67aab4344f7caaac517374e5dabd`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:10.477340 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-8eeb9cfa-fba9-4a81-9a6b-48eb44adffd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:10.543997 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 999/1998] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:10.551113 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab54b3-1ec8-4822-9505-15f809a3d48d None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:10.552775 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-21ab54b3-1ec8-4822-9505-15f809a3d48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:10.574610 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1000/1999] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:10.582019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb9cfa-fba9-4a81-9a6b-48eb44adffd0 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:10.586513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-8eeb9cfa-fba9-4a81-9a6b-48eb44adffd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:10.608110 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb9cfa-fba9-4a81-9a6b-48eb44adffd0 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:10.616635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef32469-40cd-42d7-8cb8-c6a99e1fc48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:10.617355 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef32469-40cd-42d7-8cb8-c6a99e1fc48c admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:10.617706 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef32469-40cd-42d7-8cb8-c6a99e1fc48c admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:10.618045 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef32469-40cd-42d7-8cb8-c6a99e1fc48c admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:10.619280 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef32469-40cd-42d7-8cb8-c6a99e1fc48c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:10.621581 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef32469-40cd-42d7-8cb8-c6a99e1fc48c admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:10.646682 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:10.652607 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab54b3-1ec8-4822-9505-15f809a3d48d None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:10.659359 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668abae8-ded3-4bab-b1b9-a22f2f41ff36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:10.660010 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668abae8-ded3-4bab-b1b9-a22f2f41ff36 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:10.660327 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668abae8-ded3-4bab-b1b9-a22f2f41ff36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:10.660573 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668abae8-ded3-4bab-b1b9-a22f2f41ff36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:10.663097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1001/2000] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:10 2022] POST /identity/v3/projects => generated 371 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:10.665146 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef32469-40cd-42d7-8cb8-c6a99e1fc48c admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:10.671400 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-4ef32469-40cd-42d7-8cb8-c6a99e1fc48c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:10.675267 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668abae8-ded3-4bab-b1b9-a22f2f41ff36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:10.676757 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668abae8-ded3-4bab-b1b9-a22f2f41ff36 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:10.687126 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1000/2001] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:10.687718 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668abae8-ded3-4bab-b1b9-a22f2f41ff36 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:10.698263 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668abae8-ded3-4bab-b1b9-a22f2f41ff36 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:10.741710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef32469-40cd-42d7-8cb8-c6a99e1fc48c admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:10.747366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21297480-b8e8-467b-b7b6-6c533f30ed02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:10.747811 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21297480-b8e8-467b-b7b6-6c533f30ed02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:10.748110 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21297480-b8e8-467b-b7b6-6c533f30ed02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:10.748482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21297480-b8e8-467b-b7b6-6c533f30ed02 admin admin] PATH_INFO: `/v3/projects/892540cb67fb41cda8dbb348a9ed5e8d` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:10.749865 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21297480-b8e8-467b-b7b6-6c533f30ed02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=892540cb67fb41cda8dbb348a9ed5e8d)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:10.751955 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21297480-b8e8-467b-b7b6-6c533f30ed02 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:10.763996 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb70f5-3cdc-4083-8eb5-8ef410696aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:10.764796 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb70f5-3cdc-4083-8eb5-8ef410696aed admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:10.765143 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb70f5-3cdc-4083-8eb5-8ef410696aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:10.765429 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb70f5-3cdc-4083-8eb5-8ef410696aed admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:10.766064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cb70f5-3cdc-4083-8eb5-8ef410696aed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:10.768181 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cb70f5-3cdc-4083-8eb5-8ef410696aed admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:10.779238 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:10.796443 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1001/2002] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:10 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:10.800480 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb70f5-3cdc-4083-8eb5-8ef410696aed admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:10.807124 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-21297480-b8e8-467b-b7b6-6c533f30ed02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '892540cb67fb41cda8dbb348a9ed5e8d', 'request_id': 'req-21297480-b8e8-467b-b7b6-6c533f30ed02'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:10.813002 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb70f5-3cdc-4083-8eb5-8ef410696aed admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:10.815414 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-21297480-b8e8-467b-b7b6-6c533f30ed02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '892540cb67fb41cda8dbb348a9ed5e8d', 'request_id': 'req-21297480-b8e8-467b-b7b6-6c533f30ed02'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:10.822740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-21297480-b8e8-467b-b7b6-6c533f30ed02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '892540cb67fb41cda8dbb348a9ed5e8d', 'request_id': 'req-21297480-b8e8-467b-b7b6-6c533f30ed02'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:10.834754 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-21297480-b8e8-467b-b7b6-6c533f30ed02 admin admin] The token cache is being invalidate because project 892540cb67fb41cda8dbb348a9ed5e8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:10.835041 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-21297480-b8e8-467b-b7b6-6c533f30ed02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:10.835725 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-21297480-b8e8-467b-b7b6-6c533f30ed02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:10.838491 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1002/2003] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:10 2022] DELETE /identity/v3/projects/892540cb67fb41cda8dbb348a9ed5e8d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:10.843917 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb06541-8244-4fb4-ad43-35a37b28143c None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:10.843917 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb06541-8244-4fb4-ad43-35a37b28143c None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:10.843917 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb06541-8244-4fb4-ad43-35a37b28143c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:10.866355 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-2fb06541-8244-4fb4-ad43-35a37b28143c None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:10.867673 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-2fb06541-8244-4fb4-ad43-35a37b28143c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:10.891132 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6893c847-6213-43fa-beb6-a375b46ed034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:10.891794 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893c847-6213-43fa-beb6-a375b46ed034 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:10.892080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893c847-6213-43fa-beb6-a375b46ed034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:10.892357 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893c847-6213-43fa-beb6-a375b46ed034 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:10.893107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6893c847-6213-43fa-beb6-a375b46ed034 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:10.894746 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6893c847-6213-43fa-beb6-a375b46ed034 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:10.906470 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1002/2004] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:10 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:10.910912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6893c847-6213-43fa-beb6-a375b46ed034 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:10.913568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6893c847-6213-43fa-beb6-a375b46ed034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:10.928859 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6893c847-6213-43fa-beb6-a375b46ed034 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:10.934653 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031ab69-42ed-44b9-bd6d-52cd5fd1e36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:10.935329 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031ab69-42ed-44b9-bd6d-52cd5fd1e36b admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:10.935607 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031ab69-42ed-44b9-bd6d-52cd5fd1e36b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:10.936283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031ab69-42ed-44b9-bd6d-52cd5fd1e36b admin admin] PATH_INFO: `/v3/projects/5cf49b872ceb410c84471659ddfb2291/users/6a85dce071b140938855d264334c491b/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:10.949597 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1003/2005] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:35:10.957897 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75059a9f-1d17-4434-8389-34ea308ba454 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:10.958185 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75059a9f-1d17-4434-8389-34ea308ba454 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:10.958607 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75059a9f-1d17-4434-8389-34ea308ba454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:10.959646 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a031ab69-42ed-44b9-bd6d-52cd5fd1e36b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cf49b872ceb410c84471659ddfb2291, user_id=6a85dce071b140938855d264334c491b, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:10.963347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a031ab69-42ed-44b9-bd6d-52cd5fd1e36b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:10.980537 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1003/2006] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:10 2022] PUT /identity/v3/projects/5cf49b872ceb410c84471659ddfb2291/users/6a85dce071b140938855d264334c491b/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:10.980846 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-75059a9f-1d17-4434-8389-34ea308ba454 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:10.982080 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-75059a9f-1d17-4434-8389-34ea308ba454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:10.985450 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031ab69-42ed-44b9-bd6d-52cd5fd1e36b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:11.061725 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:11.062722 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1004/2007] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:11.067034 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75059a9f-1d17-4434-8389-34ea308ba454 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:11.079010 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75059a9f-1d17-4434-8389-34ea308ba454 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:11.080173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031ab69-42ed-44b9-bd6d-52cd5fd1e36b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:11.084948 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23c092-3b38-493b-9022-ee280e8e6c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:11.085599 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23c092-3b38-493b-9022-ee280e8e6c46 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:11.085889 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23c092-3b38-493b-9022-ee280e8e6c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:11.086163 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23c092-3b38-493b-9022-ee280e8e6c46 admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:11.086844 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc20a077-327a-44e5-840d-349f83500c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:11.087498 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23c092-3b38-493b-9022-ee280e8e6c46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:35:11.088305 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23c092-3b38-493b-9022-ee280e8e6c46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:11.088686 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20a077-327a-44e5-840d-349f83500c5e admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:11.089258 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20a077-327a-44e5-840d-349f83500c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:11.089753 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20a077-327a-44e5-840d-349f83500c5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:11.090013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23c092-3b38-493b-9022-ee280e8e6c46 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:11.090780 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc20a077-327a-44e5-840d-349f83500c5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:11.093037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc20a077-327a-44e5-840d-349f83500c5e admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:11.104077 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1005/2008] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:35:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:35:11.108754 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23c092-3b38-493b-9022-ee280e8e6c46 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:11.113211 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1004/2009] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:10 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:11.117262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc20a077-327a-44e5-840d-349f83500c5e admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:11.120271 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23c092-3b38-493b-9022-ee280e8e6c46 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:11.132331 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc20a077-327a-44e5-840d-349f83500c5e admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:11.135330 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb3eac-7ee2-4f8d-b8b8-603665e7b6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:11.136482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb3eac-7ee2-4f8d-b8b8-603665e7b6a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:11.136770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb3eac-7ee2-4f8d-b8b8-603665e7b6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:11.137040 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb3eac-7ee2-4f8d-b8b8-603665e7b6a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:11.137695 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb3eac-7ee2-4f8d-b8b8-603665e7b6a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:11.139691 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb3eac-7ee2-4f8d-b8b8-603665e7b6a5 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:11.139927 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b72e3-c9da-4b50-8194-3f343bdc59b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:11.140780 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b72e3-c9da-4b50-8194-3f343bdc59b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:11.141224 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b72e3-c9da-4b50-8194-3f343bdc59b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:11.141590 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b72e3-c9da-4b50-8194-3f343bdc59b6 admin admin] PATH_INFO: `/v3/projects/5cf49b872ceb410c84471659ddfb2291/users/6a85dce071b140938855d264334c491b/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:11.144180 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0b72e3-c9da-4b50-8194-3f343bdc59b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cf49b872ceb410c84471659ddfb2291, user_id=6a85dce071b140938855d264334c491b, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:11.147368 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0b72e3-c9da-4b50-8194-3f343bdc59b6 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:11.160121 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1005/2010] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:11 2022] PUT /identity/v3/projects/5cf49b872ceb410c84471659ddfb2291/users/6a85dce071b140938855d264334c491b/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:11.183224 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1006/2011] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:11 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:35:11.184341 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b72e3-c9da-4b50-8194-3f343bdc59b6 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:11.267760 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b72e3-c9da-4b50-8194-3f343bdc59b6 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:11.339818 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d7c407-7e52-4088-926e-6cb10cb7f9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:11.340447 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7c407-7e52-4088-926e-6cb10cb7f9fa admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:11.340738 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7c407-7e52-4088-926e-6cb10cb7f9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:11.341016 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7c407-7e52-4088-926e-6cb10cb7f9fa admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:11.341710 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d7c407-7e52-4088-926e-6cb10cb7f9fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:11.344312 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d7c407-7e52-4088-926e-6cb10cb7f9fa admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:11.378680 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1006/2012] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:11 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:11.382381 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb3eac-7ee2-4f8d-b8b8-603665e7b6a5 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:11.398807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb3eac-7ee2-4f8d-b8b8-603665e7b6a5 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:11.403073 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce95be0-5cc3-404a-a40a-1d40391fd8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:11.403751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce95be0-5cc3-404a-a40a-1d40391fd8f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:11.404097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce95be0-5cc3-404a-a40a-1d40391fd8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:11.404416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce95be0-5cc3-404a-a40a-1d40391fd8f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:11.405250 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce95be0-5cc3-404a-a40a-1d40391fd8f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:11.406990 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce95be0-5cc3-404a-a40a-1d40391fd8f1 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:11.421267 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1007/2013] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:11.425002 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d7c407-7e52-4088-926e-6cb10cb7f9fa admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:11.443152 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d7c407-7e52-4088-926e-6cb10cb7f9fa admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:11.448157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4f998-0bb4-47d8-b635-14f4aafce43a None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:11.448385 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4f998-0bb4-47d8-b635-14f4aafce43a None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:11.448722 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4f998-0bb4-47d8-b635-14f4aafce43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:11.450562 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f8347-f7e7-45bf-b58b-9a4a26a76184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:11.451402 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f8347-f7e7-45bf-b58b-9a4a26a76184 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:11.451929 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f8347-f7e7-45bf-b58b-9a4a26a76184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:11.452406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f8347-f7e7-45bf-b58b-9a4a26a76184 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:11.453348 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f8347-f7e7-45bf-b58b-9a4a26a76184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:11.455133 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f8347-f7e7-45bf-b58b-9a4a26a76184 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:11.467399 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:11.469911 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1007/2014] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:11.486610 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f8347-f7e7-45bf-b58b-9a4a26a76184 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:11.492705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-27e4f998-0bb4-47d8-b635-14f4aafce43a None None] MFA Rules not processed for user `7425d9aefcf940709f3a3aad4166755c`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:11.494908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-27e4f998-0bb4-47d8-b635-14f4aafce43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:11.518423 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f8347-f7e7-45bf-b58b-9a4a26a76184 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:11.531394 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b79b5f-08f6-4eb0-bfd9-e5a7d770cbf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:11.532088 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b79b5f-08f6-4eb0-bfd9-e5a7d770cbf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:11.532379 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b79b5f-08f6-4eb0-bfd9-e5a7d770cbf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:11.532661 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b79b5f-08f6-4eb0-bfd9-e5a7d770cbf1 admin admin] PATH_INFO: `/v3/projects/312d37b4203a42229010f8c9d8db9f5d/users/1f44b9361f13454995e1505314cf5280/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:11.555208 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b79b5f-08f6-4eb0-bfd9-e5a7d770cbf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=312d37b4203a42229010f8c9d8db9f5d, user_id=1f44b9361f13454995e1505314cf5280, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:11.558772 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b79b5f-08f6-4eb0-bfd9-e5a7d770cbf1 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:11.588308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1008/2015] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:11 2022] PUT /identity/v3/projects/312d37b4203a42229010f8c9d8db9f5d/users/1f44b9361f13454995e1505314cf5280/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:11.589134 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1008/2016] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:11.602674 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ffbf8-7949-4095-8be0-db743cf969f9 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:11.602674 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ffbf8-7949-4095-8be0-db743cf969f9 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:11.602674 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ffbf8-7949-4095-8be0-db743cf969f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:11.602997 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4f998-0bb4-47d8-b635-14f4aafce43a None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:11.605540 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-27e4f998-0bb4-47d8-b635-14f4aafce43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:11.673898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-336ffbf8-7949-4095-8be0-db743cf969f9 None None] MFA Rules not processed for user `1f44b9361f13454995e1505314cf5280`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:11.675322 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-336ffbf8-7949-4095-8be0-db743cf969f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:11.701051 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4f998-0bb4-47d8-b635-14f4aafce43a None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:11.717170 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:11.758296 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af39435e-15a8-46c9-91bc-2e0ffb9cc9f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:11.758933 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af39435e-15a8-46c9-91bc-2e0ffb9cc9f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:11.759706 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af39435e-15a8-46c9-91bc-2e0ffb9cc9f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:11.759995 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af39435e-15a8-46c9-91bc-2e0ffb9cc9f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:11.769356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1009/2017] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:11.774085 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ffbf8-7949-4095-8be0-db743cf969f9 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:11.840463 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af39435e-15a8-46c9-91bc-2e0ffb9cc9f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:11.842292 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af39435e-15a8-46c9-91bc-2e0ffb9cc9f9 service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:11.849499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ffbf8-7949-4095-8be0-db743cf969f9 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:11.852202 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1009/2018] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:11 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:11.854666 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360514be-5a47-43a6-b0d4-50952cdc497b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:11.855288 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360514be-5a47-43a6-b0d4-50952cdc497b admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:11.855574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360514be-5a47-43a6-b0d4-50952cdc497b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:11.855851 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360514be-5a47-43a6-b0d4-50952cdc497b admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:11.856495 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360514be-5a47-43a6-b0d4-50952cdc497b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:11.858431 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360514be-5a47-43a6-b0d4-50952cdc497b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:11.887648 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1010/2019] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:11 2022] POST /identity/v3/projects => generated 387 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:11.892778 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af39435e-15a8-46c9-91bc-2e0ffb9cc9f9 service cinder] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:11.905671 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af39435e-15a8-46c9-91bc-2e0ffb9cc9f9 service cinder] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:11.909576 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:11.980474 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f17ca-9953-427a-9f84-9e8564bf2a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:11.981100 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f17ca-9953-427a-9f84-9e8564bf2a5b admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:11.981417 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f17ca-9953-427a-9f84-9e8564bf2a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:11.981687 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f17ca-9953-427a-9f84-9e8564bf2a5b admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:11.982393 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f17ca-9953-427a-9f84-9e8564bf2a5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:11.984473 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f17ca-9953-427a-9f84-9e8564bf2a5b admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:12.015260 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1010/2020] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:11 2022] POST /identity/v3/users => generated 294 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:12.019896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360514be-5a47-43a6-b0d4-50952cdc497b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:12.035693 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360514be-5a47-43a6-b0d4-50952cdc497b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:12.035693 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca72aa-1dfb-451a-b6be-4f8a1e67e2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:12.035693 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca72aa-1dfb-451a-b6be-4f8a1e67e2a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:12.035693 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca72aa-1dfb-451a-b6be-4f8a1e67e2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:12.045837 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca72aa-1dfb-451a-b6be-4f8a1e67e2a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:12.045837 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca72aa-1dfb-451a-b6be-4f8a1e67e2a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:12.047064 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba0de2-1955-4560-ae2d-1c024a70e969 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:12.047357 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba0de2-1955-4560-ae2d-1c024a70e969 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:12.047648 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba0de2-1955-4560-ae2d-1c024a70e969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:12.053450 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca72aa-1dfb-451a-b6be-4f8a1e67e2a5 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:12.076226 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1011/2021] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:35:12.077996 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca72aa-1dfb-451a-b6be-4f8a1e67e2a5 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:12.088978 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-85ba0de2-1955-4560-ae2d-1c024a70e969 None None] MFA Rules not processed for user `6a85dce071b140938855d264334c491b`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:12.090208 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-85ba0de2-1955-4560-ae2d-1c024a70e969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:12.091373 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca72aa-1dfb-451a-b6be-4f8a1e67e2a5 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:12.097062 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18eeb4-90de-4f6d-876c-25b292a67fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:12.097973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18eeb4-90de-4f6d-876c-25b292a67fcb admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:12.098519 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18eeb4-90de-4f6d-876c-25b292a67fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:12.099127 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18eeb4-90de-4f6d-876c-25b292a67fcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:12.100068 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18eeb4-90de-4f6d-876c-25b292a67fcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:12.102227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18eeb4-90de-4f6d-876c-25b292a67fcb admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:12.119201 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1012/2022] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:12.120253 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18eeb4-90de-4f6d-876c-25b292a67fcb admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:12.131911 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18eeb4-90de-4f6d-876c-25b292a67fcb admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:12.145738 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc083de0-55a2-4f03-b54d-3712d084760a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:12.146578 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc083de0-55a2-4f03-b54d-3712d084760a admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:12.146915 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc083de0-55a2-4f03-b54d-3712d084760a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:12.147256 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc083de0-55a2-4f03-b54d-3712d084760a admin admin] PATH_INFO: `/v3/projects/db4bd031fa4c48aaa80936f5a6c3cb23/users/826e98377fa9479e9b1892f530ae8b64/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:12.168134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc083de0-55a2-4f03-b54d-3712d084760a admin admin] RBAC: Authorizing `identity:create_grant(project_id=db4bd031fa4c48aaa80936f5a6c3cb23, user_id=826e98377fa9479e9b1892f530ae8b64, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:12.172913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc083de0-55a2-4f03-b54d-3712d084760a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:12.185674 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1011/2023] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:12 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:12.189715 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1013/2024] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:12 2022] PUT /identity/v3/projects/db4bd031fa4c48aaa80936f5a6c3cb23/users/826e98377fa9479e9b1892f530ae8b64/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:12.194178 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba0de2-1955-4560-ae2d-1c024a70e969 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:12.201599 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9781019-a427-44f6-b260-ae0216e24c69 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:12.203801 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9781019-a427-44f6-b260-ae0216e24c69 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:12.204277 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9781019-a427-44f6-b260-ae0216e24c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:12.272215 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba0de2-1955-4560-ae2d-1c024a70e969 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:12.273250 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-f9781019-a427-44f6-b260-ae0216e24c69 None None] MFA Rules not processed for user `826e98377fa9479e9b1892f530ae8b64`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:12.276175 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f9781019-a427-44f6-b260-ae0216e24c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:12.325321 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:12.356270 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b11de45-899c-4ea0-b887-7bb3c5bcd20f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:12.357001 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11de45-899c-4ea0-b887-7bb3c5bcd20f service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:12.357360 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11de45-899c-4ea0-b887-7bb3c5bcd20f service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:12.357710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11de45-899c-4ea0-b887-7bb3c5bcd20f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:12.360572 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:12.361338 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-3b11de45-899c-4ea0-b887-7bb3c5bcd20f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:12.376342 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1014/2025] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:12 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:12.377400 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9781019-a427-44f6-b260-ae0216e24c69 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:12.471588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b11de45-899c-4ea0-b887-7bb3c5bcd20f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:12.473011 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b11de45-899c-4ea0-b887-7bb3c5bcd20f service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:12.475743 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9781019-a427-44f6-b260-ae0216e24c69 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:12.480907 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9e68d-fd40-4c30-9b06-8daf104eb818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:12.481566 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9e68d-fd40-4c30-9b06-8daf104eb818 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:12.481849 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9e68d-fd40-4c30-9b06-8daf104eb818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:12.482122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9e68d-fd40-4c30-9b06-8daf104eb818 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:12.482788 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce9e68d-fd40-4c30-9b06-8daf104eb818 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:12.482951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1012/2026] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:12.484793 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce9e68d-fd40-4c30-9b06-8daf104eb818 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:12.513229 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1015/2027] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:12 2022] POST /identity/v3/projects => generated 387 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:12.517226 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b11de45-899c-4ea0-b887-7bb3c5bcd20f service cinder] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:12.529632 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b11de45-899c-4ea0-b887-7bb3c5bcd20f service cinder] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:12.604016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43a1a7-0019-444d-a15e-9d54eb9f60c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:12.604916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43a1a7-0019-444d-a15e-9d54eb9f60c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:12.605180 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43a1a7-0019-444d-a15e-9d54eb9f60c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:12.605461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43a1a7-0019-444d-a15e-9d54eb9f60c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:12.606103 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da43a1a7-0019-444d-a15e-9d54eb9f60c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:12.608423 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da43a1a7-0019-444d-a15e-9d54eb9f60c0 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:12.639731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1013/2028] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:12 2022] POST /identity/v3/users => generated 300 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:35:12.641821 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9e68d-fd40-4c30-9b06-8daf104eb818 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:12.658217 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9e68d-fd40-4c30-9b06-8daf104eb818 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:12.669669 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1a3eb-4354-428c-bbce-da04473b288f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:12.670493 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1a3eb-4354-428c-bbce-da04473b288f admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:12.670897 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1a3eb-4354-428c-bbce-da04473b288f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:12.671180 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1a3eb-4354-428c-bbce-da04473b288f admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:12.672053 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf1a3eb-4354-428c-bbce-da04473b288f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:12.675360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf1a3eb-4354-428c-bbce-da04473b288f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:12.690353 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1016/2029] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:35:12.694354 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43a1a7-0019-444d-a15e-9d54eb9f60c0 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:12.708989 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43a1a7-0019-444d-a15e-9d54eb9f60c0 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:12.714951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd90428a-cc57-4505-81fc-78fc096056a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:12.715830 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90428a-cc57-4505-81fc-78fc096056a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:12.716105 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90428a-cc57-4505-81fc-78fc096056a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:12.716431 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90428a-cc57-4505-81fc-78fc096056a9 admin admin] PATH_INFO: `/v3/projects/5714ca4786604c019f0452ed28f8581c/users/6343e8fc236449cea60fdadfdb6d7b48/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:12.737416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd90428a-cc57-4505-81fc-78fc096056a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5714ca4786604c019f0452ed28f8581c, user_id=6343e8fc236449cea60fdadfdb6d7b48, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:12.740789 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd90428a-cc57-4505-81fc-78fc096056a9 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:12.754719 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1014/2030] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:12 2022] PUT /identity/v3/projects/5714ca4786604c019f0452ed28f8581c/users/6343e8fc236449cea60fdadfdb6d7b48/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:12.760362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1a3eb-4354-428c-bbce-da04473b288f admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:12.772698 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1a3eb-4354-428c-bbce-da04473b288f admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:12.858002 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ede6e-fc95-4571-b6cd-19076b930a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:12.858658 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ede6e-fc95-4571-b6cd-19076b930a49 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:12.858936 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ede6e-fc95-4571-b6cd-19076b930a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:12.859203 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ede6e-fc95-4571-b6cd-19076b930a49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:12.859845 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ede6e-fc95-4571-b6cd-19076b930a49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:12.861499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ede6e-fc95-4571-b6cd-19076b930a49 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:12.874486 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1017/2031] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:12.877977 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd90428a-cc57-4505-81fc-78fc096056a9 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:12.889757 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd90428a-cc57-4505-81fc-78fc096056a9 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:12.895967 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d42500-2a23-4b9c-87a1-a3ea29426af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:12.896671 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d42500-2a23-4b9c-87a1-a3ea29426af0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:12.897081 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d42500-2a23-4b9c-87a1-a3ea29426af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:12.897477 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d42500-2a23-4b9c-87a1-a3ea29426af0 admin admin] PATH_INFO: `/v3/projects/5714ca4786604c019f0452ed28f8581c/users/6343e8fc236449cea60fdadfdb6d7b48/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:12.899588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d42500-2a23-4b9c-87a1-a3ea29426af0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5714ca4786604c019f0452ed28f8581c, user_id=6343e8fc236449cea60fdadfdb6d7b48, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:12.902338 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d42500-2a23-4b9c-87a1-a3ea29426af0 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:12.913078 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1015/2032] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:12 2022] PUT /identity/v3/projects/5714ca4786604c019f0452ed28f8581c/users/6343e8fc236449cea60fdadfdb6d7b48/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:12.923581 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d783c10-9a5a-44c1-b55b-8d4d75ac19a5 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:12.923873 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d783c10-9a5a-44c1-b55b-8d4d75ac19a5 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:12.924155 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d783c10-9a5a-44c1-b55b-8d4d75ac19a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:12.953334 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-8d783c10-9a5a-44c1-b55b-8d4d75ac19a5 None None] MFA Rules not processed for user `6343e8fc236449cea60fdadfdb6d7b48`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:12.954572 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8d783c10-9a5a-44c1-b55b-8d4d75ac19a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:13.003115 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:13.021492 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1018/2033] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:12 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:13.032892 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d42500-2a23-4b9c-87a1-a3ea29426af0 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:13.079164 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:13.089498 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d42500-2a23-4b9c-87a1-a3ea29426af0 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:13.093115 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36921d3f-ac8b-47ea-88c3-db668ce75aaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:13.093711 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36921d3f-ac8b-47ea-88c3-db668ce75aaf service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:13.093993 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36921d3f-ac8b-47ea-88c3-db668ce75aaf service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:13.094287 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36921d3f-ac8b-47ea-88c3-db668ce75aaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:13.103167 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36921d3f-ac8b-47ea-88c3-db668ce75aaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:13.104870 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36921d3f-ac8b-47ea-88c3-db668ce75aaf service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:13.114027 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1016/2034] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:13 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:13.368187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eada109-1d63-4898-8995-4b9caecd04f1 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:13.368552 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eada109-1d63-4898-8995-4b9caecd04f1 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:13.369540 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eada109-1d63-4898-8995-4b9caecd04f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:13.395904 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-2eada109-1d63-4898-8995-4b9caecd04f1 None None] MFA Rules not processed for user `7425d9aefcf940709f3a3aad4166755c`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:13.397224 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2eada109-1d63-4898-8995-4b9caecd04f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:13.468337 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1019/2035] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:13.821308 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36921d3f-ac8b-47ea-88c3-db668ce75aaf service cinder] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:13.831906 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36921d3f-ac8b-47ea-88c3-db668ce75aaf service cinder] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:13.835939 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c213222-dec0-4d08-9fbc-7894c647738b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:13.836594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c213222-dec0-4d08-9fbc-7894c647738b service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:13.836871 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c213222-dec0-4d08-9fbc-7894c647738b service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:13.837142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c213222-dec0-4d08-9fbc-7894c647738b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:13.934512 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c213222-dec0-4d08-9fbc-7894c647738b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:13.938076 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c213222-dec0-4d08-9fbc-7894c647738b service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:13.948404 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1017/2036] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:14.070897 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eada109-1d63-4898-8995-4b9caecd04f1 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:14.072879 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2eada109-1d63-4898-8995-4b9caecd04f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:14.113866 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eada109-1d63-4898-8995-4b9caecd04f1 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:14.120845 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820bd40-ae3c-45a8-b9c6-6c4e4121fa24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:14.124332 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820bd40-ae3c-45a8-b9c6-6c4e4121fa24 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:14.124491 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820bd40-ae3c-45a8-b9c6-6c4e4121fa24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:14.126725 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820bd40-ae3c-45a8-b9c6-6c4e4121fa24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:14.226433 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b820bd40-ae3c-45a8-b9c6-6c4e4121fa24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:14.228533 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b820bd40-ae3c-45a8-b9c6-6c4e4121fa24 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:14.246385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1020/2037] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:14 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:16.185672 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c213222-dec0-4d08-9fbc-7894c647738b service cinder] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:16.187334 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-1c213222-dec0-4d08-9fbc-7894c647738b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:16.263591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c213222-dec0-4d08-9fbc-7894c647738b service cinder] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:16.268650 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16abceb5-ea8a-4df3-8bb5-e8568ec9ae1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:16.269404 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abceb5-ea8a-4df3-8bb5-e8568ec9ae1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:16.269688 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abceb5-ea8a-4df3-8bb5-e8568ec9ae1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:16.270076 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abceb5-ea8a-4df3-8bb5-e8568ec9ae1e admin admin] PATH_INFO: `/v3/users/7ab0658d5d334fbe8100d3b340216ec2` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:16.272234 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16abceb5-ea8a-4df3-8bb5-e8568ec9ae1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ab0658d5d334fbe8100d3b340216ec2)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:16.273731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16abceb5-ea8a-4df3-8bb5-e8568ec9ae1e admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:16.333165 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-16abceb5-ea8a-4df3-8bb5-e8568ec9ae1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ab0658d5d334fbe8100d3b340216ec2'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:16.333836 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-16abceb5-ea8a-4df3-8bb5-e8568ec9ae1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ab0658d5d334fbe8100d3b340216ec2'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:16.342699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-16abceb5-ea8a-4df3-8bb5-e8568ec9ae1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ab0658d5d334fbe8100d3b340216ec2'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:16.343133 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-16abceb5-ea8a-4df3-8bb5-e8568ec9ae1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ab0658d5d334fbe8100d3b340216ec2'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:16.360358 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-16abceb5-ea8a-4df3-8bb5-e8568ec9ae1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ab0658d5d334fbe8100d3b340216ec2'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:16.380983 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1018/2038] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:16 2022] DELETE /identity/v3/users/7ab0658d5d334fbe8100d3b340216ec2 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:16.385665 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97651ec6-fc63-412b-955f-5424ac456d4b None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:16.386185 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97651ec6-fc63-412b-955f-5424ac456d4b None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:16.386519 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97651ec6-fc63-412b-955f-5424ac456d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:16.409278 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-97651ec6-fc63-412b-955f-5424ac456d4b None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:16.410625 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-97651ec6-fc63-412b-955f-5424ac456d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:16.496966 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1021/2039] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:16.506777 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16abceb5-ea8a-4df3-8bb5-e8568ec9ae1e admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:16.508382 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-16abceb5-ea8a-4df3-8bb5-e8568ec9ae1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:16.521368 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:16.582499 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16abceb5-ea8a-4df3-8bb5-e8568ec9ae1e admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:16.586123 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52fbcfe-e3fc-4a9d-8ae8-ae975bb0a4d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:16.586763 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52fbcfe-e3fc-4a9d-8ae8-ae975bb0a4d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:16.587047 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52fbcfe-e3fc-4a9d-8ae8-ae975bb0a4d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:16.587836 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52fbcfe-e3fc-4a9d-8ae8-ae975bb0a4d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:16.596750 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52fbcfe-e3fc-4a9d-8ae8-ae975bb0a4d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:16.598189 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52fbcfe-e3fc-4a9d-8ae8-ae975bb0a4d4 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:16.610909 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1019/2040] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:16.952629 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97651ec6-fc63-412b-955f-5424ac456d4b None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:16.954608 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-97651ec6-fc63-412b-955f-5424ac456d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:16.986589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97651ec6-fc63-412b-955f-5424ac456d4b None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:16.994217 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae90455-aff0-45cf-ba9d-fe2f54417a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:16.996526 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae90455-aff0-45cf-ba9d-fe2f54417a1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:16.996972 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae90455-aff0-45cf-ba9d-fe2f54417a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:16.997561 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae90455-aff0-45cf-ba9d-fe2f54417a1f admin admin] PATH_INFO: `/v3/projects/a7bf65301a364090aa54e0eea3f332c6` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:16.999266 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae90455-aff0-45cf-ba9d-fe2f54417a1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7bf65301a364090aa54e0eea3f332c6)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:17.002040 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae90455-aff0-45cf-ba9d-fe2f54417a1f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:17.054716 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:17.079981 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3ae90455-aff0-45cf-ba9d-fe2f54417a1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7bf65301a364090aa54e0eea3f332c6', 'request_id': 'req-3ae90455-aff0-45cf-ba9d-fe2f54417a1f'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:17.087933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3ae90455-aff0-45cf-ba9d-fe2f54417a1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7bf65301a364090aa54e0eea3f332c6', 'request_id': 'req-3ae90455-aff0-45cf-ba9d-fe2f54417a1f'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:17.095449 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3ae90455-aff0-45cf-ba9d-fe2f54417a1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7bf65301a364090aa54e0eea3f332c6', 'request_id': 'req-3ae90455-aff0-45cf-ba9d-fe2f54417a1f'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:17.113436 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3ae90455-aff0-45cf-ba9d-fe2f54417a1f admin admin] The token cache is being invalidate because project a7bf65301a364090aa54e0eea3f332c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:17.113826 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3ae90455-aff0-45cf-ba9d-fe2f54417a1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:17.114789 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3ae90455-aff0-45cf-ba9d-fe2f54417a1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:17.116368 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1022/2041] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:16 2022] DELETE /identity/v3/projects/a7bf65301a364090aa54e0eea3f332c6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:17.120414 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52fbcfe-e3fc-4a9d-8ae8-ae975bb0a4d4 service neutron] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:17.130120 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52fbcfe-e3fc-4a9d-8ae8-ae975bb0a4d4 service neutron] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:17.217135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73df5a-422e-442d-bc39-153737a2da8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:17.218721 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73df5a-422e-442d-bc39-153737a2da8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:17.218936 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73df5a-422e-442d-bc39-153737a2da8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:17.219220 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73df5a-422e-442d-bc39-153737a2da8c admin admin] PATH_INFO: `/v3/users/88e5060d2b43426a9c7cb0b8df042399` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:17.220403 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73df5a-422e-442d-bc39-153737a2da8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=88e5060d2b43426a9c7cb0b8df042399)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:17.222554 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73df5a-422e-442d-bc39-153737a2da8c admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:17.301197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-1f73df5a-422e-442d-bc39-153737a2da8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88e5060d2b43426a9c7cb0b8df042399'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:17.301929 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-1f73df5a-422e-442d-bc39-153737a2da8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88e5060d2b43426a9c7cb0b8df042399'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:17.311266 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-1f73df5a-422e-442d-bc39-153737a2da8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88e5060d2b43426a9c7cb0b8df042399'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:17.312034 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-1f73df5a-422e-442d-bc39-153737a2da8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88e5060d2b43426a9c7cb0b8df042399'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:17.325443 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-1f73df5a-422e-442d-bc39-153737a2da8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88e5060d2b43426a9c7cb0b8df042399'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:17.338521 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1020/2042] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:17 2022] DELETE /identity/v3/users/88e5060d2b43426a9c7cb0b8df042399 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:17.832186 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae90455-aff0-45cf-ba9d-fe2f54417a1f admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:17.834389 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3ae90455-aff0-45cf-ba9d-fe2f54417a1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:17.848495 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae90455-aff0-45cf-ba9d-fe2f54417a1f admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:17.913776 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d2532-d445-4a11-8b8d-0ab04753a62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:17.914605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d2532-d445-4a11-8b8d-0ab04753a62a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:17.914909 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d2532-d445-4a11-8b8d-0ab04753a62a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:17.915185 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d2532-d445-4a11-8b8d-0ab04753a62a admin admin] PATH_INFO: `/v3/projects/3aaedec85cd54cfcbe5da73bc3290127` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:17.916398 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d2532-d445-4a11-8b8d-0ab04753a62a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3aaedec85cd54cfcbe5da73bc3290127)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:17.918635 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d2532-d445-4a11-8b8d-0ab04753a62a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:17.966028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-048d2532-d445-4a11-8b8d-0ab04753a62a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3aaedec85cd54cfcbe5da73bc3290127', 'request_id': 'req-048d2532-d445-4a11-8b8d-0ab04753a62a'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:17.972118 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-048d2532-d445-4a11-8b8d-0ab04753a62a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3aaedec85cd54cfcbe5da73bc3290127', 'request_id': 'req-048d2532-d445-4a11-8b8d-0ab04753a62a'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:17.980033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-048d2532-d445-4a11-8b8d-0ab04753a62a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3aaedec85cd54cfcbe5da73bc3290127', 'request_id': 'req-048d2532-d445-4a11-8b8d-0ab04753a62a'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:17.991196 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-048d2532-d445-4a11-8b8d-0ab04753a62a admin admin] The token cache is being invalidate because project 3aaedec85cd54cfcbe5da73bc3290127 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:17.991499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-048d2532-d445-4a11-8b8d-0ab04753a62a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:17.992408 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-048d2532-d445-4a11-8b8d-0ab04753a62a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:17.994024 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1023/2043] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:17 2022] DELETE /identity/v3/projects/3aaedec85cd54cfcbe5da73bc3290127 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:17.998693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7f502-8bda-4415-8a1e-a45c345a6ef4 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:17.999003 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7f502-8bda-4415-8a1e-a45c345a6ef4 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:17.999359 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7f502-8bda-4415-8a1e-a45c345a6ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:18.020771 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-73e7f502-8bda-4415-8a1e-a45c345a6ef4 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:18.022158 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-73e7f502-8bda-4415-8a1e-a45c345a6ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:18.043994 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:18.087345 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1021/2044] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:18.095328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1f498-b8d2-4719-91fc-673260a9547c None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:18.095689 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1f498-b8d2-4719-91fc-673260a9547c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:18.096037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1f498-b8d2-4719-91fc-673260a9547c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:18.117257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-43c1f498-b8d2-4719-91fc-673260a9547c None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:18.118999 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-43c1f498-b8d2-4719-91fc-673260a9547c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:18.127245 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1024/2045] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:18.131730 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e7f502-8bda-4415-8a1e-a45c345a6ef4 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:18.142099 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e7f502-8bda-4415-8a1e-a45c345a6ef4 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:18.150512 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e5370d-c2e5-493a-809d-162a774ab096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:18.151207 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5370d-c2e5-493a-809d-162a774ab096 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:18.151491 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5370d-c2e5-493a-809d-162a774ab096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:18.151765 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5370d-c2e5-493a-809d-162a774ab096 admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:18.152442 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e5370d-c2e5-493a-809d-162a774ab096 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:35:18.152742 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e5370d-c2e5-493a-809d-162a774ab096 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:18.154373 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e5370d-c2e5-493a-809d-162a774ab096 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:18.166478 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1022/2046] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:35:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:35:18.170728 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c1f498-b8d2-4719-91fc-673260a9547c None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:18.180030 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c1f498-b8d2-4719-91fc-673260a9547c None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:18.185208 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2270e84-32ec-413b-a2c5-101a35b6cbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:18.185812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2270e84-32ec-413b-a2c5-101a35b6cbde admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:18.186655 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2270e84-32ec-413b-a2c5-101a35b6cbde admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:18.186655 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2270e84-32ec-413b-a2c5-101a35b6cbde admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:18.187056 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2270e84-32ec-413b-a2c5-101a35b6cbde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:18.188957 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2270e84-32ec-413b-a2c5-101a35b6cbde admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:18.210984 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e5370d-c2e5-493a-809d-162a774ab096 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:18.213130 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-c7e5370d-c2e5-493a-809d-162a774ab096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:18.220072 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1025/2047] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:18 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:18.224821 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2270e84-32ec-413b-a2c5-101a35b6cbde admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:18.244516 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:18.277931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e5370d-c2e5-493a-809d-162a774ab096 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:18.291653 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2270e84-32ec-413b-a2c5-101a35b6cbde admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:18.296474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a0731-323a-4474-9e6d-c2cb26b349be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:18.297092 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a0731-323a-4474-9e6d-c2cb26b349be admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:18.297375 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a0731-323a-4474-9e6d-c2cb26b349be admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:18.297650 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a0731-323a-4474-9e6d-c2cb26b349be admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:18.298306 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a0731-323a-4474-9e6d-c2cb26b349be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:18.300245 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a0731-323a-4474-9e6d-c2cb26b349be admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:18.332352 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1026/2048] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:18 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:35:18.334192 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab114b84-bbf5-49ae-b4ab-9f802b86197f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:18.334563 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a0731-323a-4474-9e6d-c2cb26b349be admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:18.335079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab114b84-bbf5-49ae-b4ab-9f802b86197f service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:18.335468 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab114b84-bbf5-49ae-b4ab-9f802b86197f service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:18.335756 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab114b84-bbf5-49ae-b4ab-9f802b86197f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:18.337579 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-ab114b84-bbf5-49ae-b4ab-9f802b86197f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:18.346048 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a0731-323a-4474-9e6d-c2cb26b349be admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:18.363360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36459a-85d7-4ea8-9dae-56d92862dd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:18.363996 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36459a-85d7-4ea8-9dae-56d92862dd12 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:18.364289 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36459a-85d7-4ea8-9dae-56d92862dd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:18.364705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36459a-85d7-4ea8-9dae-56d92862dd12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:18.365492 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e36459a-85d7-4ea8-9dae-56d92862dd12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:18.367158 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e36459a-85d7-4ea8-9dae-56d92862dd12 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:18.379900 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1027/2049] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:18.385613 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36459a-85d7-4ea8-9dae-56d92862dd12 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:18.397006 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36459a-85d7-4ea8-9dae-56d92862dd12 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:18.402093 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a8215e-15fe-44d5-807a-684cdde1e222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:18.402773 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a8215e-15fe-44d5-807a-684cdde1e222 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:18.403266 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a8215e-15fe-44d5-807a-684cdde1e222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:18.403383 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a8215e-15fe-44d5-807a-684cdde1e222 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:18.404015 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a8215e-15fe-44d5-807a-684cdde1e222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:18.405899 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a8215e-15fe-44d5-807a-684cdde1e222 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:18.414357 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab114b84-bbf5-49ae-b4ab-9f802b86197f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:18.417580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab114b84-bbf5-49ae-b4ab-9f802b86197f service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:18.418151 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1028/2050] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:18.422223 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a8215e-15fe-44d5-807a-684cdde1e222 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:18.428660 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1023/2051] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:18 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:35:18.436167 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a8215e-15fe-44d5-807a-684cdde1e222 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:18.441082 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b0b0c1-6160-41a5-8b25-6b461701ff9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:18.441945 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0b0c1-6160-41a5-8b25-6b461701ff9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:18.442444 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0b0c1-6160-41a5-8b25-6b461701ff9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:18.442742 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0b0c1-6160-41a5-8b25-6b461701ff9b admin admin] PATH_INFO: `/v3/projects/464634e60473484aa0d8fe4613e9ef3b/users/af50319a58204845a7d369fbd8455f7b/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:18.467355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b0b0c1-6160-41a5-8b25-6b461701ff9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=464634e60473484aa0d8fe4613e9ef3b, user_id=af50319a58204845a7d369fbd8455f7b, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:18.470331 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b0b0c1-6160-41a5-8b25-6b461701ff9b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:18.486038 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1029/2052] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:18 2022] PUT /identity/v3/projects/464634e60473484aa0d8fe4613e9ef3b/users/af50319a58204845a7d369fbd8455f7b/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:18.497904 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d60b43a-2bc1-41d3-9df0-e4477ebab7d7 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:18.498378 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d60b43a-2bc1-41d3-9df0-e4477ebab7d7 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:18.498837 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d60b43a-2bc1-41d3-9df0-e4477ebab7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:18.530681 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-1d60b43a-2bc1-41d3-9df0-e4477ebab7d7 None None] MFA Rules not processed for user `af50319a58204845a7d369fbd8455f7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:18.532095 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-1d60b43a-2bc1-41d3-9df0-e4477ebab7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:18.547198 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:18.625655 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1024/2053] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:18.629900 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b0b0c1-6160-41a5-8b25-6b461701ff9b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:18.705589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b0b0c1-6160-41a5-8b25-6b461701ff9b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:18.710686 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb05f41-2739-4969-b00a-e489bc24f50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:18.711376 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb05f41-2739-4969-b00a-e489bc24f50e admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:18.711730 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb05f41-2739-4969-b00a-e489bc24f50e admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:18.712125 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb05f41-2739-4969-b00a-e489bc24f50e admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:18.712829 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb05f41-2739-4969-b00a-e489bc24f50e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:18.714898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb05f41-2739-4969-b00a-e489bc24f50e admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:18.750466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1030/2054] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:18 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:18.754830 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d60b43a-2bc1-41d3-9df0-e4477ebab7d7 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:18.768198 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d60b43a-2bc1-41d3-9df0-e4477ebab7d7 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:18.871794 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27833e-6416-41ba-9479-c120d65ffc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:18.872551 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27833e-6416-41ba-9479-c120d65ffc73 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:18.872905 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27833e-6416-41ba-9479-c120d65ffc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:18.873254 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27833e-6416-41ba-9479-c120d65ffc73 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:18.873979 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27833e-6416-41ba-9479-c120d65ffc73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:18.876173 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27833e-6416-41ba-9479-c120d65ffc73 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:18.908068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1025/2055] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:18 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:18.912192 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb05f41-2739-4969-b00a-e489bc24f50e admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:18.925696 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb05f41-2739-4969-b00a-e489bc24f50e admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:18.933619 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ac101-f7f2-48bc-b000-7de14e03d93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:18.934409 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ac101-f7f2-48bc-b000-7de14e03d93a admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:18.934577 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ac101-f7f2-48bc-b000-7de14e03d93a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:18.934854 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ac101-f7f2-48bc-b000-7de14e03d93a admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:18.935547 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ac101-f7f2-48bc-b000-7de14e03d93a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:18.937407 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ac101-f7f2-48bc-b000-7de14e03d93a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:18.953374 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1031/2056] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:18.959908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27833e-6416-41ba-9479-c120d65ffc73 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:18.972525 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27833e-6416-41ba-9479-c120d65ffc73 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:18.977212 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eed9cc-c804-4c0a-93ba-0f81860fc5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:18.977941 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eed9cc-c804-4c0a-93ba-0f81860fc5d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:18.978226 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eed9cc-c804-4c0a-93ba-0f81860fc5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:18.978561 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eed9cc-c804-4c0a-93ba-0f81860fc5d6 admin admin] PATH_INFO: `/v3/projects/123c0d192b8c421181e474685054559d/users/d805891917574c8cb1229cb267c3cb12/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:18.994471 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eed9cc-c804-4c0a-93ba-0f81860fc5d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=123c0d192b8c421181e474685054559d, user_id=d805891917574c8cb1229cb267c3cb12, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:18.997142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eed9cc-c804-4c0a-93ba-0f81860fc5d6 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:19.010160 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1026/2057] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:18 2022] PUT /identity/v3/projects/123c0d192b8c421181e474685054559d/users/d805891917574c8cb1229cb267c3cb12/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:19.014214 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ac101-f7f2-48bc-b000-7de14e03d93a admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:19.024331 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ac101-f7f2-48bc-b000-7de14e03d93a admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:19.057962 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:19.090607 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65207375-7dd5-4783-aad1-9801f68dafa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:19.091222 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65207375-7dd5-4783-aad1-9801f68dafa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:19.091506 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65207375-7dd5-4783-aad1-9801f68dafa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:19.091783 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65207375-7dd5-4783-aad1-9801f68dafa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:19.092519 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65207375-7dd5-4783-aad1-9801f68dafa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:19.094459 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65207375-7dd5-4783-aad1-9801f68dafa4 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:19.107532 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1032/2058] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:19.112140 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eed9cc-c804-4c0a-93ba-0f81860fc5d6 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:19.124769 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eed9cc-c804-4c0a-93ba-0f81860fc5d6 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:19.145732 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6617c1da-dfc5-49df-8787-c54d9a00bb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:19.146453 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617c1da-dfc5-49df-8787-c54d9a00bb49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:19.146856 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617c1da-dfc5-49df-8787-c54d9a00bb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:19.147136 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617c1da-dfc5-49df-8787-c54d9a00bb49 admin admin] PATH_INFO: `/v3/projects/123c0d192b8c421181e474685054559d/users/d805891917574c8cb1229cb267c3cb12/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:19.150425 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6617c1da-dfc5-49df-8787-c54d9a00bb49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=123c0d192b8c421181e474685054559d, user_id=d805891917574c8cb1229cb267c3cb12, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:19.153689 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6617c1da-dfc5-49df-8787-c54d9a00bb49 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:19.165102 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1027/2059] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:19 2022] PUT /identity/v3/projects/123c0d192b8c421181e474685054559d/users/d805891917574c8cb1229cb267c3cb12/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:19.177108 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5f726-5039-42b0-a8e7-06e0cb0dfc24 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:19.177401 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5f726-5039-42b0-a8e7-06e0cb0dfc24 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:19.177682 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5f726-5039-42b0-a8e7-06e0cb0dfc24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:19.234760 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-e2c5f726-5039-42b0-a8e7-06e0cb0dfc24 None None] MFA Rules not processed for user `d805891917574c8cb1229cb267c3cb12`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:19.236014 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-e2c5f726-5039-42b0-a8e7-06e0cb0dfc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:19.314791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1033/2060] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:19.328120 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6617c1da-dfc5-49df-8787-c54d9a00bb49 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:19.416146 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6617c1da-dfc5-49df-8787-c54d9a00bb49 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:19.421032 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf2421b-aee0-4de4-99ac-4b68d2e4d011 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:19.421866 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf2421b-aee0-4de4-99ac-4b68d2e4d011 service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:19.422219 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf2421b-aee0-4de4-99ac-4b68d2e4d011 service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:19.422622 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf2421b-aee0-4de4-99ac-4b68d2e4d011 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:19.433542 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf2421b-aee0-4de4-99ac-4b68d2e4d011 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:19.435267 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf2421b-aee0-4de4-99ac-4b68d2e4d011 service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:19.447120 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1028/2061] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:20.069113 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c5f726-5039-42b0-a8e7-06e0cb0dfc24 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:20.153091 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c5f726-5039-42b0-a8e7-06e0cb0dfc24 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:20.166524 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6f458-9538-442c-abf3-578a0149a4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:20.166524 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6f458-9538-442c-abf3-578a0149a4ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:20.166524 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6f458-9538-442c-abf3-578a0149a4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:20.166524 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6f458-9538-442c-abf3-578a0149a4ea admin admin] PATH_INFO: `/v3/users/af50319a58204845a7d369fbd8455f7b` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:20.166925 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c6f458-9538-442c-abf3-578a0149a4ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=af50319a58204845a7d369fbd8455f7b)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:20.166925 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c6f458-9538-442c-abf3-578a0149a4ea admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:20.222095 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c4c6f458-9538-442c-abf3-578a0149a4ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af50319a58204845a7d369fbd8455f7b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:20.233355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c4c6f458-9538-442c-abf3-578a0149a4ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af50319a58204845a7d369fbd8455f7b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:20.234378 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c4c6f458-9538-442c-abf3-578a0149a4ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af50319a58204845a7d369fbd8455f7b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:20.236933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:20.252371 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c4c6f458-9538-442c-abf3-578a0149a4ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af50319a58204845a7d369fbd8455f7b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:20.253706 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-c4c6f458-9538-442c-abf3-578a0149a4ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af50319a58204845a7d369fbd8455f7b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:20.268719 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1034/2062] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:20 2022] DELETE /identity/v3/users/af50319a58204845a7d369fbd8455f7b => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:20.273242 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e81e70-a305-444f-a426-d14249c19ff0 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:20.273796 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e81e70-a305-444f-a426-d14249c19ff0 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:20.274310 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e81e70-a305-444f-a426-d14249c19ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:20.296611 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-f0e81e70-a305-444f-a426-d14249c19ff0 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:20.297847 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-f0e81e70-a305-444f-a426-d14249c19ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:20.302714 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:20.379375 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1029/2063] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:35:20.394293 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6f458-9538-442c-abf3-578a0149a4ea admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:20.394293 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c4c6f458-9538-442c-abf3-578a0149a4ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:20.456995 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6f458-9538-442c-abf3-578a0149a4ea admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:20.460489 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca3bca-0f96-4d0e-95df-94af27089628 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:20.461135 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca3bca-0f96-4d0e-95df-94af27089628 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:20.461423 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca3bca-0f96-4d0e-95df-94af27089628 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:20.461701 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca3bca-0f96-4d0e-95df-94af27089628 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:20.472301 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca3bca-0f96-4d0e-95df-94af27089628 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:20.473805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca3bca-0f96-4d0e-95df-94af27089628 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:20.485663 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1035/2064] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:35:20.768877 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e81e70-a305-444f-a426-d14249c19ff0 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:20.770501 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-f0e81e70-a305-444f-a426-d14249c19ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:20.784644 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e81e70-a305-444f-a426-d14249c19ff0 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:20.789967 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40b04f-ef54-4894-9e58-c0085ed04b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:20.790637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40b04f-ef54-4894-9e58-c0085ed04b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:20.790918 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40b04f-ef54-4894-9e58-c0085ed04b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:20.791187 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40b04f-ef54-4894-9e58-c0085ed04b73 admin admin] PATH_INFO: `/v3/projects/464634e60473484aa0d8fe4613e9ef3b` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:20.792312 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c40b04f-ef54-4894-9e58-c0085ed04b73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=464634e60473484aa0d8fe4613e9ef3b)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:20.794213 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c40b04f-ef54-4894-9e58-c0085ed04b73 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:20.844163 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-6c40b04f-ef54-4894-9e58-c0085ed04b73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '464634e60473484aa0d8fe4613e9ef3b', 'request_id': 'req-6c40b04f-ef54-4894-9e58-c0085ed04b73'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:20.853170 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-6c40b04f-ef54-4894-9e58-c0085ed04b73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '464634e60473484aa0d8fe4613e9ef3b', 'request_id': 'req-6c40b04f-ef54-4894-9e58-c0085ed04b73'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:20.861021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-6c40b04f-ef54-4894-9e58-c0085ed04b73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '464634e60473484aa0d8fe4613e9ef3b', 'request_id': 'req-6c40b04f-ef54-4894-9e58-c0085ed04b73'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:20.871610 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-6c40b04f-ef54-4894-9e58-c0085ed04b73 admin admin] The token cache is being invalidate because project 464634e60473484aa0d8fe4613e9ef3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:20.871907 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-6c40b04f-ef54-4894-9e58-c0085ed04b73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:20.872697 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-6c40b04f-ef54-4894-9e58-c0085ed04b73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:20.874196 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1030/2065] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:20 2022] DELETE /identity/v3/projects/464634e60473484aa0d8fe4613e9ef3b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:20.878167 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca3bca-0f96-4d0e-95df-94af27089628 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:20.888405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca3bca-0f96-4d0e-95df-94af27089628 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:20.937315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40b04f-ef54-4894-9e58-c0085ed04b73 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:20.938975 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-6c40b04f-ef54-4894-9e58-c0085ed04b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:20.950721 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7843f857-4992-4f80-abcc-9249d6694fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:20.951363 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843f857-4992-4f80-abcc-9249d6694fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:20.951650 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843f857-4992-4f80-abcc-9249d6694fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:20.951929 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843f857-4992-4f80-abcc-9249d6694fa4 admin admin] PATH_INFO: `/v3/users/d805891917574c8cb1229cb267c3cb12` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:20.953094 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7843f857-4992-4f80-abcc-9249d6694fa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d805891917574c8cb1229cb267c3cb12)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:20.955281 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7843f857-4992-4f80-abcc-9249d6694fa4 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:21.004703 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7843f857-4992-4f80-abcc-9249d6694fa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd805891917574c8cb1229cb267c3cb12'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:21.006584 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40b04f-ef54-4894-9e58-c0085ed04b73 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:21.012371 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef227c48-f3ee-4df0-abdd-6f1dfe645731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:21.013068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef227c48-f3ee-4df0-abdd-6f1dfe645731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:21.013377 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7843f857-4992-4f80-abcc-9249d6694fa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd805891917574c8cb1229cb267c3cb12'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:21.013520 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef227c48-f3ee-4df0-abdd-6f1dfe645731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:21.013667 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef227c48-f3ee-4df0-abdd-6f1dfe645731 admin admin] PATH_INFO: `/v3/users/1f44b9361f13454995e1505314cf5280` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:21.014253 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7843f857-4992-4f80-abcc-9249d6694fa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd805891917574c8cb1229cb267c3cb12'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:21.014798 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef227c48-f3ee-4df0-abdd-6f1dfe645731 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f44b9361f13454995e1505314cf5280)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:21.016732 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef227c48-f3ee-4df0-abdd-6f1dfe645731 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:21.027312 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7843f857-4992-4f80-abcc-9249d6694fa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd805891917574c8cb1229cb267c3cb12'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:21.027992 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7843f857-4992-4f80-abcc-9249d6694fa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd805891917574c8cb1229cb267c3cb12'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:21.033386 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:21.042905 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1036/2066] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:20 2022] DELETE /identity/v3/users/d805891917574c8cb1229cb267c3cb12 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:21.059465 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:21.072806 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-ef227c48-f3ee-4df0-abdd-6f1dfe645731 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f44b9361f13454995e1505314cf5280'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:21.073690 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-ef227c48-f3ee-4df0-abdd-6f1dfe645731 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f44b9361f13454995e1505314cf5280'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:21.082963 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-ef227c48-f3ee-4df0-abdd-6f1dfe645731 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f44b9361f13454995e1505314cf5280'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:21.083884 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-ef227c48-f3ee-4df0-abdd-6f1dfe645731 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f44b9361f13454995e1505314cf5280'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:21.104393 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-ef227c48-f3ee-4df0-abdd-6f1dfe645731 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f44b9361f13454995e1505314cf5280'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:21.118084 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1031/2067] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:20 2022] DELETE /identity/v3/users/1f44b9361f13454995e1505314cf5280 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:21.123224 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4615113-b8c7-4a64-a0d2-24ff6ddc8b17 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:21.123679 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4615113-b8c7-4a64-a0d2-24ff6ddc8b17 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:21.123781 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4615113-b8c7-4a64-a0d2-24ff6ddc8b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:21.147288 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-c4615113-b8c7-4a64-a0d2-24ff6ddc8b17 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:21.148585 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c4615113-b8c7-4a64-a0d2-24ff6ddc8b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:21.242015 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1037/2068] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:21.254209 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef227c48-f3ee-4df0-abdd-6f1dfe645731 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:21.255761 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-ef227c48-f3ee-4df0-abdd-6f1dfe645731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:21.335638 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef227c48-f3ee-4df0-abdd-6f1dfe645731 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:21.374143 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971083b-d419-4129-808f-09f9cb82ebea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:21.375866 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971083b-d419-4129-808f-09f9cb82ebea service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:21.376213 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971083b-d419-4129-808f-09f9cb82ebea service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:21.376498 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971083b-d419-4129-808f-09f9cb82ebea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:21.390756 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c971083b-d419-4129-808f-09f9cb82ebea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:21.392643 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c971083b-d419-4129-808f-09f9cb82ebea service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:21.404969 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1032/2069] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:21.645180 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4615113-b8c7-4a64-a0d2-24ff6ddc8b17 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:21.647235 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c4615113-b8c7-4a64-a0d2-24ff6ddc8b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:21.663654 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4615113-b8c7-4a64-a0d2-24ff6ddc8b17 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:21.669214 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abbfe64-910b-4723-9ce5-0b65f6655cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:21.669952 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbfe64-910b-4723-9ce5-0b65f6655cfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:21.671232 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbfe64-910b-4723-9ce5-0b65f6655cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:21.671581 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbfe64-910b-4723-9ce5-0b65f6655cfa admin admin] PATH_INFO: `/v3/projects/123c0d192b8c421181e474685054559d` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:21.672820 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abbfe64-910b-4723-9ce5-0b65f6655cfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=123c0d192b8c421181e474685054559d)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:21.675777 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abbfe64-910b-4723-9ce5-0b65f6655cfa admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:21.745598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0abbfe64-910b-4723-9ce5-0b65f6655cfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '123c0d192b8c421181e474685054559d', 'request_id': 'req-0abbfe64-910b-4723-9ce5-0b65f6655cfa'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:21.754871 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0abbfe64-910b-4723-9ce5-0b65f6655cfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '123c0d192b8c421181e474685054559d', 'request_id': 'req-0abbfe64-910b-4723-9ce5-0b65f6655cfa'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:21.764517 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0abbfe64-910b-4723-9ce5-0b65f6655cfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '123c0d192b8c421181e474685054559d', 'request_id': 'req-0abbfe64-910b-4723-9ce5-0b65f6655cfa'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:21.781600 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0abbfe64-910b-4723-9ce5-0b65f6655cfa admin admin] The token cache is being invalidate because project 123c0d192b8c421181e474685054559d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:21.782036 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0abbfe64-910b-4723-9ce5-0b65f6655cfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:21.782806 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0abbfe64-910b-4723-9ce5-0b65f6655cfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:21.784616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1038/2070] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:21 2022] DELETE /identity/v3/projects/123c0d192b8c421181e474685054559d => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:21.789799 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973ce95-ec53-4cc4-8299-2fd31feec94d None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:21.790698 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973ce95-ec53-4cc4-8299-2fd31feec94d None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:21.790698 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973ce95-ec53-4cc4-8299-2fd31feec94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:21.813089 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-e973ce95-ec53-4cc4-8299-2fd31feec94d None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:21.814589 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-e973ce95-ec53-4cc4-8299-2fd31feec94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:21.851695 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abbfe64-910b-4723-9ce5-0b65f6655cfa admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:21.853585 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0abbfe64-910b-4723-9ce5-0b65f6655cfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:21.898909 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1033/2071] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:21.906462 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca8f0f-a6a8-46e5-aa51-8ca2ba81066a None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:21.906757 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca8f0f-a6a8-46e5-aa51-8ca2ba81066a None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:21.907031 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca8f0f-a6a8-46e5-aa51-8ca2ba81066a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:21.928530 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-f4ca8f0f-a6a8-46e5-aa51-8ca2ba81066a None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:21.929797 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-f4ca8f0f-a6a8-46e5-aa51-8ca2ba81066a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:21.942941 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1034/2072] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:35:21.944407 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ca8f0f-a6a8-46e5-aa51-8ca2ba81066a None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:21.950021 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abbfe64-910b-4723-9ce5-0b65f6655cfa admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:21.955081 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa4559-f754-44de-ac3b-d1a74e66202a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:21.955314 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ca8f0f-a6a8-46e5-aa51-8ca2ba81066a None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:21.955817 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa4559-f754-44de-ac3b-d1a74e66202a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:21.956200 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa4559-f754-44de-ac3b-d1a74e66202a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:21.956882 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa4559-f754-44de-ac3b-d1a74e66202a admin admin] PATH_INFO: `/v3/projects/312d37b4203a42229010f8c9d8db9f5d` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:21.957605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa4559-f754-44de-ac3b-d1a74e66202a admin admin] RBAC: Authorizing `identity:delete_project(project_id=312d37b4203a42229010f8c9d8db9f5d)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:21.959536 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa4559-f754-44de-ac3b-d1a74e66202a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:21.960599 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c8e46c-e342-48a3-ac95-81bee87cf05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:21.961278 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8e46c-e342-48a3-ac95-81bee87cf05a admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:21.961679 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8e46c-e342-48a3-ac95-81bee87cf05a admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:21.962076 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8e46c-e342-48a3-ac95-81bee87cf05a admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:21.962987 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c8e46c-e342-48a3-ac95-81bee87cf05a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:35:21.963371 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c8e46c-e342-48a3-ac95-81bee87cf05a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:21.965690 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c8e46c-e342-48a3-ac95-81bee87cf05a admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:21.984552 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1035/2073] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:35:21.985899 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c8e46c-e342-48a3-ac95-81bee87cf05a admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:21.986850 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:21.999106 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c8e46c-e342-48a3-ac95-81bee87cf05a admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:22.004007 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a8403-c0f7-4da7-8216-c3507c7b3c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:22.004742 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a8403-c0f7-4da7-8216-c3507c7b3c82 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:22.005024 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a8403-c0f7-4da7-8216-c3507c7b3c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:22.005302 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a8403-c0f7-4da7-8216-c3507c7b3c82 admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:22.005933 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a8403-c0f7-4da7-8216-c3507c7b3c82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:22.007894 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a8403-c0f7-4da7-8216-c3507c7b3c82 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:22.015045 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f2aa4559-f754-44de-ac3b-d1a74e66202a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '312d37b4203a42229010f8c9d8db9f5d', 'request_id': 'req-f2aa4559-f754-44de-ac3b-d1a74e66202a'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:22.021352 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f2aa4559-f754-44de-ac3b-d1a74e66202a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '312d37b4203a42229010f8c9d8db9f5d', 'request_id': 'req-f2aa4559-f754-44de-ac3b-d1a74e66202a'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:22.029870 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f2aa4559-f754-44de-ac3b-d1a74e66202a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '312d37b4203a42229010f8c9d8db9f5d', 'request_id': 'req-f2aa4559-f754-44de-ac3b-d1a74e66202a'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:22.036855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1036/2074] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:21 2022] POST /identity/v3/projects => generated 363 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:22.041745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a8403-c0f7-4da7-8216-c3507c7b3c82 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:22.042671 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f2aa4559-f754-44de-ac3b-d1a74e66202a admin admin] The token cache is being invalidate because project 312d37b4203a42229010f8c9d8db9f5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:22.042972 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f2aa4559-f754-44de-ac3b-d1a74e66202a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:22.043642 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f2aa4559-f754-44de-ac3b-d1a74e66202a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:22.045221 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1039/2075] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:21 2022] DELETE /identity/v3/projects/312d37b4203a42229010f8c9d8db9f5d => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:22.048867 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa4559-f754-44de-ac3b-d1a74e66202a admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:22.050265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f2aa4559-f754-44de-ac3b-d1a74e66202a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:22.055040 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:22.107389 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a8403-c0f7-4da7-8216-c3507c7b3c82 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:22.112092 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c223146-4e1b-45a4-8502-25cbad3bfb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:22.112696 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c223146-4e1b-45a4-8502-25cbad3bfb9e admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:22.112981 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c223146-4e1b-45a4-8502-25cbad3bfb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:22.113306 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c223146-4e1b-45a4-8502-25cbad3bfb9e admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:22.113890 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c223146-4e1b-45a4-8502-25cbad3bfb9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:22.115865 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c223146-4e1b-45a4-8502-25cbad3bfb9e admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:22.116656 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa4559-f754-44de-ac3b-d1a74e66202a admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:22.121309 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3268cb-1434-477d-8003-04147ee264e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:22.121936 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3268cb-1434-477d-8003-04147ee264e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:22.122218 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3268cb-1434-477d-8003-04147ee264e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:22.122518 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3268cb-1434-477d-8003-04147ee264e9 admin admin] PATH_INFO: `/v3/users/826e98377fa9479e9b1892f530ae8b64` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:22.123685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3268cb-1434-477d-8003-04147ee264e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=826e98377fa9479e9b1892f530ae8b64)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:22.125599 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3268cb-1434-477d-8003-04147ee264e9 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:22.137137 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1037/2076] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:22 2022] POST /identity/v3/users => generated 282 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:35:22.141312 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c223146-4e1b-45a4-8502-25cbad3bfb9e admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:22.142962 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-2c223146-4e1b-45a4-8502-25cbad3bfb9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:22.155479 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c223146-4e1b-45a4-8502-25cbad3bfb9e admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:22.161323 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22589d-6dc3-49a1-a0f1-ba79cd3f42b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:22.162137 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22589d-6dc3-49a1-a0f1-ba79cd3f42b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:22.162462 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22589d-6dc3-49a1-a0f1-ba79cd3f42b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:22.162740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22589d-6dc3-49a1-a0f1-ba79cd3f42b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:22.163441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa22589d-6dc3-49a1-a0f1-ba79cd3f42b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:22.165051 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa22589d-6dc3-49a1-a0f1-ba79cd3f42b9 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:22.173735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5e3268cb-1434-477d-8003-04147ee264e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '826e98377fa9479e9b1892f530ae8b64'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:22.175755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1038/2077] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:22.180074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22589d-6dc3-49a1-a0f1-ba79cd3f42b9 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:22.181773 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5e3268cb-1434-477d-8003-04147ee264e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '826e98377fa9479e9b1892f530ae8b64'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:22.182404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5e3268cb-1434-477d-8003-04147ee264e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '826e98377fa9479e9b1892f530ae8b64'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:22.189533 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22589d-6dc3-49a1-a0f1-ba79cd3f42b9 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:22.194598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5e0e0-1b0f-4597-b10e-e6a7a50b2c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:22.195013 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5e3268cb-1434-477d-8003-04147ee264e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '826e98377fa9479e9b1892f530ae8b64'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:22.195173 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5e0e0-1b0f-4597-b10e-e6a7a50b2c8f admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:22.195449 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5e0e0-1b0f-4597-b10e-e6a7a50b2c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:22.195686 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5e3268cb-1434-477d-8003-04147ee264e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '826e98377fa9479e9b1892f530ae8b64'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:22.195782 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5e0e0-1b0f-4597-b10e-e6a7a50b2c8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:22.196854 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e5e0e0-1b0f-4597-b10e-e6a7a50b2c8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:22.198459 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e5e0e0-1b0f-4597-b10e-e6a7a50b2c8f admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:22.208462 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1040/2078] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:22 2022] DELETE /identity/v3/users/826e98377fa9479e9b1892f530ae8b64 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:22.208829 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1039/2079] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:35:22.213342 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3268cb-1434-477d-8003-04147ee264e9 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:22.214762 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-5e3268cb-1434-477d-8003-04147ee264e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:22.248495 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:22.288843 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3268cb-1434-477d-8003-04147ee264e9 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:22.294916 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8453e-8b49-49c7-bcb6-a9f8ea1a6125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:22.296104 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8453e-8b49-49c7-bcb6-a9f8ea1a6125 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:22.296412 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8453e-8b49-49c7-bcb6-a9f8ea1a6125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:22.296695 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8453e-8b49-49c7-bcb6-a9f8ea1a6125 admin admin] PATH_INFO: `/v3/projects/d08d4f0eff884afeb9ed9d14884ed91b/users/9af9f4e78871455580738ed3ff112fe1/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:22.312428 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b8453e-8b49-49c7-bcb6-a9f8ea1a6125 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d08d4f0eff884afeb9ed9d14884ed91b, user_id=9af9f4e78871455580738ed3ff112fe1, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:22.315269 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b8453e-8b49-49c7-bcb6-a9f8ea1a6125 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:22.328695 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1041/2080] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:22 2022] PUT /identity/v3/projects/d08d4f0eff884afeb9ed9d14884ed91b/users/9af9f4e78871455580738ed3ff112fe1/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:22.339455 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ccf25-f580-4933-93cb-60916ce85bcc None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:22.339833 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ccf25-f580-4933-93cb-60916ce85bcc None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:22.340105 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ccf25-f580-4933-93cb-60916ce85bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:22.383011 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-758ccf25-f580-4933-93cb-60916ce85bcc None None] MFA Rules not processed for user `9af9f4e78871455580738ed3ff112fe1`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:22.384271 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-758ccf25-f580-4933-93cb-60916ce85bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:22.452179 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1040/2081] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:22.457233 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8453e-8b49-49c7-bcb6-a9f8ea1a6125 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:22.499813 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ccf25-f580-4933-93cb-60916ce85bcc None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:22.501281 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-758ccf25-f580-4933-93cb-60916ce85bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:22.563375 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8453e-8b49-49c7-bcb6-a9f8ea1a6125 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:22.588689 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412e794-7b31-4286-b793-11186d6c419e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:22.589466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412e794-7b31-4286-b793-11186d6c419e admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:22.589757 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412e794-7b31-4286-b793-11186d6c419e admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:22.590166 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412e794-7b31-4286-b793-11186d6c419e admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:22.591213 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412e794-7b31-4286-b793-11186d6c419e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:22.593067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412e794-7b31-4286-b793-11186d6c419e admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:22.598192 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ccf25-f580-4933-93cb-60916ce85bcc None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:22.603069 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155976a9-a2ac-4853-9637-36a9c5eef402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:22.604247 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155976a9-a2ac-4853-9637-36a9c5eef402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:22.604533 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155976a9-a2ac-4853-9637-36a9c5eef402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:22.604804 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155976a9-a2ac-4853-9637-36a9c5eef402 admin admin] PATH_INFO: `/v3/projects/db4bd031fa4c48aaa80936f5a6c3cb23` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:22.606462 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155976a9-a2ac-4853-9637-36a9c5eef402 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db4bd031fa4c48aaa80936f5a6c3cb23)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:22.608378 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155976a9-a2ac-4853-9637-36a9c5eef402 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:22.622948 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:22.647064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1042/2082] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:22 2022] POST /identity/v3/projects => generated 363 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:22.648122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412e794-7b31-4286-b793-11186d6c419e admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:22.667142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-155976a9-a2ac-4853-9637-36a9c5eef402 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db4bd031fa4c48aaa80936f5a6c3cb23', 'request_id': 'req-155976a9-a2ac-4853-9637-36a9c5eef402'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:22.677620 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-155976a9-a2ac-4853-9637-36a9c5eef402 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db4bd031fa4c48aaa80936f5a6c3cb23', 'request_id': 'req-155976a9-a2ac-4853-9637-36a9c5eef402'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:22.686627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-155976a9-a2ac-4853-9637-36a9c5eef402 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db4bd031fa4c48aaa80936f5a6c3cb23', 'request_id': 'req-155976a9-a2ac-4853-9637-36a9c5eef402'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:22.702007 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-155976a9-a2ac-4853-9637-36a9c5eef402 admin admin] The token cache is being invalidate because project db4bd031fa4c48aaa80936f5a6c3cb23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:22.702351 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-155976a9-a2ac-4853-9637-36a9c5eef402 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:22.703073 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-155976a9-a2ac-4853-9637-36a9c5eef402 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:22.704834 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1041/2083] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:22 2022] DELETE /identity/v3/projects/db4bd031fa4c48aaa80936f5a6c3cb23 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:22.717341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155976a9-a2ac-4853-9637-36a9c5eef402 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:22.719601 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-155976a9-a2ac-4853-9637-36a9c5eef402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:22.732512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412e794-7b31-4286-b793-11186d6c419e admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:22.817835 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:22.844665 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9edf35-70fd-4c71-905d-e332d5b31700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:22.845283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9edf35-70fd-4c71-905d-e332d5b31700 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:22.845838 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9edf35-70fd-4c71-905d-e332d5b31700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:22.845959 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9edf35-70fd-4c71-905d-e332d5b31700 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:22.846645 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9edf35-70fd-4c71-905d-e332d5b31700 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:22.848702 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9edf35-70fd-4c71-905d-e332d5b31700 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:22.885753 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155976a9-a2ac-4853-9637-36a9c5eef402 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:22.889374 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1043/2084] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:22 2022] POST /identity/v3/users => generated 288 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:35:22.892646 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9edf35-70fd-4c71-905d-e332d5b31700 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:22.894711 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-db9edf35-70fd-4c71-905d-e332d5b31700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:22.895948 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622505f1-24f0-4c57-b204-5ea39cc44dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:22.896656 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622505f1-24f0-4c57-b204-5ea39cc44dba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:22.896943 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622505f1-24f0-4c57-b204-5ea39cc44dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:22.897222 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622505f1-24f0-4c57-b204-5ea39cc44dba admin admin] PATH_INFO: `/v3/users/6343e8fc236449cea60fdadfdb6d7b48` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:22.898804 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622505f1-24f0-4c57-b204-5ea39cc44dba admin admin] RBAC: Authorizing `identity:delete_user(user_id=6343e8fc236449cea60fdadfdb6d7b48)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:22.902226 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622505f1-24f0-4c57-b204-5ea39cc44dba admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:22.908246 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9edf35-70fd-4c71-905d-e332d5b31700 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:22.913516 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ecf276-31f2-419b-9bc1-3d7b91e71f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:22.914307 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ecf276-31f2-419b-9bc1-3d7b91e71f94 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:22.914656 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ecf276-31f2-419b-9bc1-3d7b91e71f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:22.915007 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ecf276-31f2-419b-9bc1-3d7b91e71f94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:22.916286 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ecf276-31f2-419b-9bc1-3d7b91e71f94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:22.917875 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ecf276-31f2-419b-9bc1-3d7b91e71f94 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:22.929286 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1044/2085] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:22.935500 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ecf276-31f2-419b-9bc1-3d7b91e71f94 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:22.957804 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ecf276-31f2-419b-9bc1-3d7b91e71f94 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:22.963320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e57bc4-0169-4066-9b13-4a8b5ab9d75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:22.964106 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e57bc4-0169-4066-9b13-4a8b5ab9d75b admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:22.964396 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e57bc4-0169-4066-9b13-4a8b5ab9d75b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:22.965038 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e57bc4-0169-4066-9b13-4a8b5ab9d75b admin admin] PATH_INFO: `/v3/projects/61fda57cc5a74312b6322e0805ad9708/users/f0cb8dae00734bb38fd3c3331a043685/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:22.970631 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-622505f1-24f0-4c57-b204-5ea39cc44dba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6343e8fc236449cea60fdadfdb6d7b48'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:22.971484 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-622505f1-24f0-4c57-b204-5ea39cc44dba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6343e8fc236449cea60fdadfdb6d7b48'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:22.980861 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e57bc4-0169-4066-9b13-4a8b5ab9d75b admin admin] RBAC: Authorizing `identity:create_grant(project_id=61fda57cc5a74312b6322e0805ad9708, user_id=f0cb8dae00734bb38fd3c3331a043685, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:22.983692 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e57bc4-0169-4066-9b13-4a8b5ab9d75b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:22.984271 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-622505f1-24f0-4c57-b204-5ea39cc44dba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6343e8fc236449cea60fdadfdb6d7b48'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:22.985070 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-622505f1-24f0-4c57-b204-5ea39cc44dba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6343e8fc236449cea60fdadfdb6d7b48'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:22.997962 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1045/2086] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:22 2022] PUT /identity/v3/projects/61fda57cc5a74312b6322e0805ad9708/users/f0cb8dae00734bb38fd3c3331a043685/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:23.001663 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e57bc4-0169-4066-9b13-4a8b5ab9d75b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:23.003271 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b4e57bc4-0169-4066-9b13-4a8b5ab9d75b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:23.013750 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-622505f1-24f0-4c57-b204-5ea39cc44dba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6343e8fc236449cea60fdadfdb6d7b48'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:23.028518 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1042/2087] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:22 2022] DELETE /identity/v3/users/6343e8fc236449cea60fdadfdb6d7b48 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:23.070203 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e57bc4-0169-4066-9b13-4a8b5ab9d75b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:23.135513 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c439c-ffbb-4fd6-af02-68ece5013cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:23.136166 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c439c-ffbb-4fd6-af02-68ece5013cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:23.136455 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c439c-ffbb-4fd6-af02-68ece5013cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:23.136747 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c439c-ffbb-4fd6-af02-68ece5013cc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:23.137402 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c439c-ffbb-4fd6-af02-68ece5013cc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:23.139505 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c439c-ffbb-4fd6-af02-68ece5013cc7 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:23.151503 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1046/2088] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:35:23.155647 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622505f1-24f0-4c57-b204-5ea39cc44dba admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:23.166075 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622505f1-24f0-4c57-b204-5ea39cc44dba admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:23.174954 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b71b5-9dec-42e1-82c6-9a3f32f5fbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:23.175647 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b71b5-9dec-42e1-82c6-9a3f32f5fbea admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:23.175931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b71b5-9dec-42e1-82c6-9a3f32f5fbea admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:23.176208 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b71b5-9dec-42e1-82c6-9a3f32f5fbea admin admin] PATH_INFO: `/v3/projects/61fda57cc5a74312b6322e0805ad9708/users/f0cb8dae00734bb38fd3c3331a043685/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:23.178290 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b71b5-9dec-42e1-82c6-9a3f32f5fbea admin admin] RBAC: Authorizing `identity:create_grant(project_id=61fda57cc5a74312b6322e0805ad9708, user_id=f0cb8dae00734bb38fd3c3331a043685, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:23.180968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b71b5-9dec-42e1-82c6-9a3f32f5fbea admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:23.193239 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1043/2089] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:23 2022] PUT /identity/v3/projects/61fda57cc5a74312b6322e0805ad9708/users/f0cb8dae00734bb38fd3c3331a043685/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:23.204899 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba8e25-0920-4ab2-a83b-746648db8726 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:23.205190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba8e25-0920-4ab2-a83b-746648db8726 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:23.205467 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba8e25-0920-4ab2-a83b-746648db8726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:23.236195 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-37ba8e25-0920-4ab2-a83b-746648db8726 None None] MFA Rules not processed for user `f0cb8dae00734bb38fd3c3331a043685`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:23.237658 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-37ba8e25-0920-4ab2-a83b-746648db8726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:23.282351 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:23.332882 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1047/2090] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:23 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:23.340001 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b71b5-9dec-42e1-82c6-9a3f32f5fbea admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:23.341773 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-eb9b71b5-9dec-42e1-82c6-9a3f32f5fbea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:23.347063 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba8e25-0920-4ab2-a83b-746648db8726 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:23.354914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-37ba8e25-0920-4ab2-a83b-746648db8726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:23.361895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:23.423933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba8e25-0920-4ab2-a83b-746648db8726 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:23.428227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf16f5a4-d1f5-422a-b63e-625c80427887 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:23.428849 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf16f5a4-d1f5-422a-b63e-625c80427887 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:23.429129 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf16f5a4-d1f5-422a-b63e-625c80427887 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:23.429408 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf16f5a4-d1f5-422a-b63e-625c80427887 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:23.431441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b71b5-9dec-42e1-82c6-9a3f32f5fbea admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:23.436264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c09fa4-9488-41d7-8f37-a6bfe0394c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:23.437133 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c09fa4-9488-41d7-8f37-a6bfe0394c70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:23.437513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c09fa4-9488-41d7-8f37-a6bfe0394c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:23.437885 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c09fa4-9488-41d7-8f37-a6bfe0394c70 admin admin] PATH_INFO: `/v3/projects/5714ca4786604c019f0452ed28f8581c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:23.439119 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c09fa4-9488-41d7-8f37-a6bfe0394c70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5714ca4786604c019f0452ed28f8581c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:23.439619 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf16f5a4-d1f5-422a-b63e-625c80427887 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:23.441056 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf16f5a4-d1f5-422a-b63e-625c80427887 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:23.441177 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c09fa4-9488-41d7-8f37-a6bfe0394c70 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:23.451450 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1048/2091] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:23 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:35:23.495842 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-39c09fa4-9488-41d7-8f37-a6bfe0394c70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5714ca4786604c019f0452ed28f8581c', 'request_id': 'req-39c09fa4-9488-41d7-8f37-a6bfe0394c70'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:23.505414 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-39c09fa4-9488-41d7-8f37-a6bfe0394c70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5714ca4786604c019f0452ed28f8581c', 'request_id': 'req-39c09fa4-9488-41d7-8f37-a6bfe0394c70'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:23.514953 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-39c09fa4-9488-41d7-8f37-a6bfe0394c70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5714ca4786604c019f0452ed28f8581c', 'request_id': 'req-39c09fa4-9488-41d7-8f37-a6bfe0394c70'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:23.529347 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-39c09fa4-9488-41d7-8f37-a6bfe0394c70 admin admin] The token cache is being invalidate because project 5714ca4786604c019f0452ed28f8581c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:23.529719 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-39c09fa4-9488-41d7-8f37-a6bfe0394c70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:23.530522 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-39c09fa4-9488-41d7-8f37-a6bfe0394c70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:23.532088 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1044/2092] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:23 2022] DELETE /identity/v3/projects/5714ca4786604c019f0452ed28f8581c => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:23.538003 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d95edb-b496-45f4-8aea-7cb3ba3d30e8 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:23.538322 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d95edb-b496-45f4-8aea-7cb3ba3d30e8 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:23.538603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d95edb-b496-45f4-8aea-7cb3ba3d30e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:23.559308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-b8d95edb-b496-45f4-8aea-7cb3ba3d30e8 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:23.560523 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b8d95edb-b496-45f4-8aea-7cb3ba3d30e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:23.624111 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1049/2093] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:23.632973 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19521e46-992a-49ef-b4da-ba7de63e36fe None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:23.633352 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19521e46-992a-49ef-b4da-ba7de63e36fe None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:23.633626 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19521e46-992a-49ef-b4da-ba7de63e36fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:23.654013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-19521e46-992a-49ef-b4da-ba7de63e36fe None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:23.655273 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-19521e46-992a-49ef-b4da-ba7de63e36fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:23.663517 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1045/2094] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:23.671486 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d95edb-b496-45f4-8aea-7cb3ba3d30e8 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:23.683362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d95edb-b496-45f4-8aea-7cb3ba3d30e8 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:23.689431 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe7bb3-1b47-4bf4-b7b8-65f6a2677895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:23.690368 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe7bb3-1b47-4bf4-b7b8-65f6a2677895 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:23.690902 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe7bb3-1b47-4bf4-b7b8-65f6a2677895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:23.691388 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe7bb3-1b47-4bf4-b7b8-65f6a2677895 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:23.692153 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe7bb3-1b47-4bf4-b7b8-65f6a2677895 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:35:23.692935 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe7bb3-1b47-4bf4-b7b8-65f6a2677895 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:23.694600 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe7bb3-1b47-4bf4-b7b8-65f6a2677895 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:23.708442 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1050/2095] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:35:23.715323 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19521e46-992a-49ef-b4da-ba7de63e36fe None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:23.725555 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19521e46-992a-49ef-b4da-ba7de63e36fe None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:23.732864 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc00ec9-bb61-427f-befe-010ba481b153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:23.733483 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc00ec9-bb61-427f-befe-010ba481b153 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:23.733766 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc00ec9-bb61-427f-befe-010ba481b153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:23.734043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc00ec9-bb61-427f-befe-010ba481b153 admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:23.734736 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc00ec9-bb61-427f-befe-010ba481b153 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:23.737003 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc00ec9-bb61-427f-befe-010ba481b153 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:23.777060 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1046/2096] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:23 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:23.778181 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe7bb3-1b47-4bf4-b7b8-65f6a2677895 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:23.788352 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe7bb3-1b47-4bf4-b7b8-65f6a2677895 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:23.903456 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:23.921404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a482fd2-7764-493c-98ac-8ca61804bd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:23.926146 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a482fd2-7764-493c-98ac-8ca61804bd56 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:23.927512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a482fd2-7764-493c-98ac-8ca61804bd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:23.927889 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a482fd2-7764-493c-98ac-8ca61804bd56 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:23.928668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a482fd2-7764-493c-98ac-8ca61804bd56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:23.932251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a482fd2-7764-493c-98ac-8ca61804bd56 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:23.969202 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1051/2097] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:23 2022] POST /identity/v3/users => generated 289 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:35:23.974126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc00ec9-bb61-427f-befe-010ba481b153 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:23.989226 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a482fd2-7764-493c-98ac-8ca61804bd56 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:23.995180 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8a482fd2-7764-493c-98ac-8ca61804bd56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:23.995452 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc00ec9-bb61-427f-befe-010ba481b153 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:24.018917 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98df394-0e32-426d-bb08-38bb04508ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:24.022138 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98df394-0e32-426d-bb08-38bb04508ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:24.023064 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98df394-0e32-426d-bb08-38bb04508ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:24.024899 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98df394-0e32-426d-bb08-38bb04508ea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:24.026444 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98df394-0e32-426d-bb08-38bb04508ea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:24.032712 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98df394-0e32-426d-bb08-38bb04508ea5 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:24.041485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a482fd2-7764-493c-98ac-8ca61804bd56 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:24.054833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254dba0a-892a-4ddd-b338-51c3a260fa9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:24.056272 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254dba0a-892a-4ddd-b338-51c3a260fa9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:24.056682 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254dba0a-892a-4ddd-b338-51c3a260fa9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:24.057474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254dba0a-892a-4ddd-b338-51c3a260fa9e admin admin] PATH_INFO: `/v3/users/9af9f4e78871455580738ed3ff112fe1` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:24.059902 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254dba0a-892a-4ddd-b338-51c3a260fa9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9af9f4e78871455580738ed3ff112fe1)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:24.064220 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1047/2098] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:24.065527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98df394-0e32-426d-bb08-38bb04508ea5 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:24.066037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254dba0a-892a-4ddd-b338-51c3a260fa9e admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:24.091849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98df394-0e32-426d-bb08-38bb04508ea5 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:24.101968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7185e-fc9c-4b95-82b9-5b576ab53433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:24.104237 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7185e-fc9c-4b95-82b9-5b576ab53433 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:24.104535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7185e-fc9c-4b95-82b9-5b576ab53433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:24.104807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7185e-fc9c-4b95-82b9-5b576ab53433 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:24.106287 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7185e-fc9c-4b95-82b9-5b576ab53433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:24.109921 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7185e-fc9c-4b95-82b9-5b576ab53433 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:24.130593 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1048/2099] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:24.139609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7185e-fc9c-4b95-82b9-5b576ab53433 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:24.176801 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7185e-fc9c-4b95-82b9-5b576ab53433 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:24.178472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-254dba0a-892a-4ddd-b338-51c3a260fa9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9af9f4e78871455580738ed3ff112fe1'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:24.183764 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8094e201-0d20-40a3-b28d-68a92493dcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:24.186728 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094e201-0d20-40a3-b28d-68a92493dcef admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:24.187006 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094e201-0d20-40a3-b28d-68a92493dcef admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:24.187456 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094e201-0d20-40a3-b28d-68a92493dcef admin admin] PATH_INFO: `/v3/projects/a9be0a80b8664b3da5467a729ba4061c/users/439b001718a047528f2f670c39b3e87d/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:24.203008 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-254dba0a-892a-4ddd-b338-51c3a260fa9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9af9f4e78871455580738ed3ff112fe1'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:24.204986 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-254dba0a-892a-4ddd-b338-51c3a260fa9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9af9f4e78871455580738ed3ff112fe1'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:24.224711 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-254dba0a-892a-4ddd-b338-51c3a260fa9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9af9f4e78871455580738ed3ff112fe1'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:24.225203 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8094e201-0d20-40a3-b28d-68a92493dcef admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9be0a80b8664b3da5467a729ba4061c, user_id=439b001718a047528f2f670c39b3e87d, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:24.225694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-254dba0a-892a-4ddd-b338-51c3a260fa9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9af9f4e78871455580738ed3ff112fe1'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:24.228784 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8094e201-0d20-40a3-b28d-68a92493dcef admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:24.242940 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1052/2100] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:23 2022] DELETE /identity/v3/users/9af9f4e78871455580738ed3ff112fe1 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:24.247925 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080ff83-cb73-4a99-bd64-347f64bc7c7f None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:24.248474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080ff83-cb73-4a99-bd64-347f64bc7c7f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:24.248997 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080ff83-cb73-4a99-bd64-347f64bc7c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:24.258338 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1049/2101] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:24 2022] PUT /identity/v3/projects/a9be0a80b8664b3da5467a729ba4061c/users/439b001718a047528f2f670c39b3e87d/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:24.267803 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a33dd-5022-4b41-b4d7-8dfd4e07d8a0 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:24.268354 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a33dd-5022-4b41-b4d7-8dfd4e07d8a0 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:24.268843 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a33dd-5022-4b41-b4d7-8dfd4e07d8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:24.289890 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-6080ff83-cb73-4a99-bd64-347f64bc7c7f None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:24.291487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6080ff83-cb73-4a99-bd64-347f64bc7c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:24.322710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-492a33dd-5022-4b41-b4d7-8dfd4e07d8a0 None None] MFA Rules not processed for user `439b001718a047528f2f670c39b3e87d`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:24.324724 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-492a33dd-5022-4b41-b4d7-8dfd4e07d8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:24.359466 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:24.425577 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1053/2102] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:24.449088 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6080ff83-cb73-4a99-bd64-347f64bc7c7f None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:24.451494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6080ff83-cb73-4a99-bd64-347f64bc7c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:24.479859 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1050/2103] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:35:24.494651 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a33dd-5022-4b41-b4d7-8dfd4e07d8a0 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:24.496156 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-492a33dd-5022-4b41-b4d7-8dfd4e07d8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:24.529670 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6080ff83-cb73-4a99-bd64-347f64bc7c7f None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:24.534014 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12848984-20a2-471d-bb86-76b03d73299d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:24.534688 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12848984-20a2-471d-bb86-76b03d73299d service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:24.534998 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12848984-20a2-471d-bb86-76b03d73299d service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:24.535247 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12848984-20a2-471d-bb86-76b03d73299d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:24.546522 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12848984-20a2-471d-bb86-76b03d73299d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:24.547808 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12848984-20a2-471d-bb86-76b03d73299d service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:24.559968 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1054/2104] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:24.583074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a33dd-5022-4b41-b4d7-8dfd4e07d8a0 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:24.592558 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b83a303-c108-4815-bd7b-52d2e72df36a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:24.593465 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b83a303-c108-4815-bd7b-52d2e72df36a service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:24.593881 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b83a303-c108-4815-bd7b-52d2e72df36a service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:24.596013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b83a303-c108-4815-bd7b-52d2e72df36a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:24.607765 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b83a303-c108-4815-bd7b-52d2e72df36a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:24.609971 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b83a303-c108-4815-bd7b-52d2e72df36a service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:24.620981 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1051/2105] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:35:24.893287 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12848984-20a2-471d-bb86-76b03d73299d service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:24.894775 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-12848984-20a2-471d-bb86-76b03d73299d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:24.906685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:24.914426 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12848984-20a2-471d-bb86-76b03d73299d service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:24.933534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb081f6-85b4-4dfc-8e0c-64bd1a618a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:24.934253 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb081f6-85b4-4dfc-8e0c-64bd1a618a9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:24.934610 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb081f6-85b4-4dfc-8e0c-64bd1a618a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:24.934898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb081f6-85b4-4dfc-8e0c-64bd1a618a9c admin admin] PATH_INFO: `/v3/projects/d08d4f0eff884afeb9ed9d14884ed91b` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:24.936430 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb081f6-85b4-4dfc-8e0c-64bd1a618a9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d08d4f0eff884afeb9ed9d14884ed91b)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:24.941638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb081f6-85b4-4dfc-8e0c-64bd1a618a9c admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:25.024961 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9bb081f6-85b4-4dfc-8e0c-64bd1a618a9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd08d4f0eff884afeb9ed9d14884ed91b', 'request_id': 'req-9bb081f6-85b4-4dfc-8e0c-64bd1a618a9c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:25.036834 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9bb081f6-85b4-4dfc-8e0c-64bd1a618a9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd08d4f0eff884afeb9ed9d14884ed91b', 'request_id': 'req-9bb081f6-85b4-4dfc-8e0c-64bd1a618a9c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:25.049782 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9bb081f6-85b4-4dfc-8e0c-64bd1a618a9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd08d4f0eff884afeb9ed9d14884ed91b', 'request_id': 'req-9bb081f6-85b4-4dfc-8e0c-64bd1a618a9c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:25.068575 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9bb081f6-85b4-4dfc-8e0c-64bd1a618a9c admin admin] The token cache is being invalidate because project d08d4f0eff884afeb9ed9d14884ed91b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:25.068908 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9bb081f6-85b4-4dfc-8e0c-64bd1a618a9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:25.069630 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9bb081f6-85b4-4dfc-8e0c-64bd1a618a9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:25.072556 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1055/2106] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:24 2022] DELETE /identity/v3/projects/d08d4f0eff884afeb9ed9d14884ed91b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:25.076585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b83a303-c108-4815-bd7b-52d2e72df36a service cinder] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:25.090867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b83a303-c108-4815-bd7b-52d2e72df36a service cinder] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:25.161788 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5121998-3cfe-41bd-9648-17efe10f20ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:25.162546 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5121998-3cfe-41bd-9648-17efe10f20ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:25.162896 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5121998-3cfe-41bd-9648-17efe10f20ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:25.163237 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5121998-3cfe-41bd-9648-17efe10f20ea admin admin] PATH_INFO: `/v3/users/f0cb8dae00734bb38fd3c3331a043685` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:25.164473 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5121998-3cfe-41bd-9648-17efe10f20ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0cb8dae00734bb38fd3c3331a043685)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:25.167064 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5121998-3cfe-41bd-9648-17efe10f20ea admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:25.224608 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-b5121998-3cfe-41bd-9648-17efe10f20ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0cb8dae00734bb38fd3c3331a043685'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:25.225357 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-b5121998-3cfe-41bd-9648-17efe10f20ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0cb8dae00734bb38fd3c3331a043685'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:25.234875 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-b5121998-3cfe-41bd-9648-17efe10f20ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0cb8dae00734bb38fd3c3331a043685'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:25.235508 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-b5121998-3cfe-41bd-9648-17efe10f20ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0cb8dae00734bb38fd3c3331a043685'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:25.238321 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:25.263175 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-b5121998-3cfe-41bd-9648-17efe10f20ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0cb8dae00734bb38fd3c3331a043685'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:25.279466 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1052/2107] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:25 2022] DELETE /identity/v3/users/f0cb8dae00734bb38fd3c3331a043685 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:25.663675 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb081f6-85b4-4dfc-8e0c-64bd1a618a9c admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:25.666074 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9bb081f6-85b4-4dfc-8e0c-64bd1a618a9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:25.685721 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb081f6-85b4-4dfc-8e0c-64bd1a618a9c admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:25.757672 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2cd14b-45ac-409d-a6ce-5cccf9adbf55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:25.758526 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2cd14b-45ac-409d-a6ce-5cccf9adbf55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:25.758906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2cd14b-45ac-409d-a6ce-5cccf9adbf55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:25.759247 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2cd14b-45ac-409d-a6ce-5cccf9adbf55 admin admin] PATH_INFO: `/v3/projects/61fda57cc5a74312b6322e0805ad9708` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:25.760529 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2cd14b-45ac-409d-a6ce-5cccf9adbf55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61fda57cc5a74312b6322e0805ad9708)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:25.762619 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2cd14b-45ac-409d-a6ce-5cccf9adbf55 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:25.813345 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9e2cd14b-45ac-409d-a6ce-5cccf9adbf55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61fda57cc5a74312b6322e0805ad9708', 'request_id': 'req-9e2cd14b-45ac-409d-a6ce-5cccf9adbf55'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:25.820261 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9e2cd14b-45ac-409d-a6ce-5cccf9adbf55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61fda57cc5a74312b6322e0805ad9708', 'request_id': 'req-9e2cd14b-45ac-409d-a6ce-5cccf9adbf55'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:25.828199 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9e2cd14b-45ac-409d-a6ce-5cccf9adbf55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61fda57cc5a74312b6322e0805ad9708', 'request_id': 'req-9e2cd14b-45ac-409d-a6ce-5cccf9adbf55'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:25.840100 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9e2cd14b-45ac-409d-a6ce-5cccf9adbf55 admin admin] The token cache is being invalidate because project 61fda57cc5a74312b6322e0805ad9708 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:25.840392 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9e2cd14b-45ac-409d-a6ce-5cccf9adbf55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:25.841027 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9e2cd14b-45ac-409d-a6ce-5cccf9adbf55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:25.842646 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1056/2108] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:25 2022] DELETE /identity/v3/projects/61fda57cc5a74312b6322e0805ad9708 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:25.849007 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7572c4b-0428-4452-af61-88a08847c76d None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:25.849503 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7572c4b-0428-4452-af61-88a08847c76d None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:25.849598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7572c4b-0428-4452-af61-88a08847c76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:25.875466 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-b7572c4b-0428-4452-af61-88a08847c76d None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:25.887234 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-b7572c4b-0428-4452-af61-88a08847c76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:25.948720 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1053/2109] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:25.956297 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0483a-80ce-42f0-8ad8-c64e15887623 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:25.956635 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0483a-80ce-42f0-8ad8-c64e15887623 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:25.956860 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0483a-80ce-42f0-8ad8-c64e15887623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:25.977610 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-15e0483a-80ce-42f0-8ad8-c64e15887623 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:25.979496 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-15e0483a-80ce-42f0-8ad8-c64e15887623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:25.987567 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1057/2110] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:25.991824 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7572c4b-0428-4452-af61-88a08847c76d None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:26.002779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7572c4b-0428-4452-af61-88a08847c76d None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:26.017038 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6289b9-0776-4bf2-92b2-ca420c9d5c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:26.017730 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6289b9-0776-4bf2-92b2-ca420c9d5c37 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:26.018220 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6289b9-0776-4bf2-92b2-ca420c9d5c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:26.018624 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6289b9-0776-4bf2-92b2-ca420c9d5c37 admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:26.019467 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6289b9-0776-4bf2-92b2-ca420c9d5c37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:35:26.019840 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6289b9-0776-4bf2-92b2-ca420c9d5c37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:26.021843 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6289b9-0776-4bf2-92b2-ca420c9d5c37 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:26.042275 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1054/2111] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:35:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:35:26.043424 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0483a-80ce-42f0-8ad8-c64e15887623 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:26.055364 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0483a-80ce-42f0-8ad8-c64e15887623 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:26.062188 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9aaa7-9e52-46c2-85de-c1b3abe110fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:26.063556 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9aaa7-9e52-46c2-85de-c1b3abe110fb admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:26.063947 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9aaa7-9e52-46c2-85de-c1b3abe110fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:26.064351 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9aaa7-9e52-46c2-85de-c1b3abe110fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:26.065221 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d9aaa7-9e52-46c2-85de-c1b3abe110fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:26.067442 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d9aaa7-9e52-46c2-85de-c1b3abe110fb admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:26.075869 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:26.107891 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1058/2112] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:26 2022] POST /identity/v3/projects => generated 391 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:26.112188 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6289b9-0776-4bf2-92b2-ca420c9d5c37 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:26.126309 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6289b9-0776-4bf2-92b2-ca420c9d5c37 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:26.191441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ee6c0-ddd2-4f7b-8a8b-bf2e8e2c9a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:26.192056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ee6c0-ddd2-4f7b-8a8b-bf2e8e2c9a4f admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:26.192363 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ee6c0-ddd2-4f7b-8a8b-bf2e8e2c9a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:26.192641 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ee6c0-ddd2-4f7b-8a8b-bf2e8e2c9a4f admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:26.193281 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161ee6c0-ddd2-4f7b-8a8b-bf2e8e2c9a4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:26.195348 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161ee6c0-ddd2-4f7b-8a8b-bf2e8e2c9a4f admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:26.218480 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1055/2113] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:26 2022] POST /identity/v3/users => generated 296 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:35:26.222483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9aaa7-9e52-46c2-85de-c1b3abe110fb admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:26.235831 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9aaa7-9e52-46c2-85de-c1b3abe110fb admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:26.241129 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ccb61-ebe0-4d36-a568-02ba664be81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:26.241866 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ccb61-ebe0-4d36-a568-02ba664be81f admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:26.242206 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ccb61-ebe0-4d36-a568-02ba664be81f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:26.242619 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ccb61-ebe0-4d36-a568-02ba664be81f admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:26.243416 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157ccb61-ebe0-4d36-a568-02ba664be81f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:26.245099 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157ccb61-ebe0-4d36-a568-02ba664be81f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:26.261384 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1059/2114] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:26.262689 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ee6c0-ddd2-4f7b-8a8b-bf2e8e2c9a4f admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:26.272935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ee6c0-ddd2-4f7b-8a8b-bf2e8e2c9a4f admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:26.278560 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba8a18d-6481-4ef4-a8d6-08cc383a6d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:26.279246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba8a18d-6481-4ef4-a8d6-08cc383a6d59 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:26.279563 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba8a18d-6481-4ef4-a8d6-08cc383a6d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:26.279853 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba8a18d-6481-4ef4-a8d6-08cc383a6d59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:26.280501 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba8a18d-6481-4ef4-a8d6-08cc383a6d59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:26.282102 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba8a18d-6481-4ef4-a8d6-08cc383a6d59 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:26.292784 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1056/2115] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:26.296943 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ccb61-ebe0-4d36-a568-02ba664be81f admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:26.308196 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ccb61-ebe0-4d36-a568-02ba664be81f admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:26.313250 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd859cd-3b4f-4966-97c9-a2f276428fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:26.313837 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd859cd-3b4f-4966-97c9-a2f276428fa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:26.314159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd859cd-3b4f-4966-97c9-a2f276428fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:26.314501 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd859cd-3b4f-4966-97c9-a2f276428fa4 admin admin] PATH_INFO: `/v3/projects/908c30c0feaf418ab8b991eb441ba2e8/users/1754d78f57bd4973ade282eedf52ad2d/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:26.331967 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd859cd-3b4f-4966-97c9-a2f276428fa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=908c30c0feaf418ab8b991eb441ba2e8, user_id=1754d78f57bd4973ade282eedf52ad2d, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:26.334739 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd859cd-3b4f-4966-97c9-a2f276428fa4 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:26.348242 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1060/2116] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:26 2022] PUT /identity/v3/projects/908c30c0feaf418ab8b991eb441ba2e8/users/1754d78f57bd4973ade282eedf52ad2d/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:26.359953 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c4e47-db27-4251-9c93-8eeff4771941 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:26.361065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c4e47-db27-4251-9c93-8eeff4771941 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:26.361451 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c4e47-db27-4251-9c93-8eeff4771941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:26.402036 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-b73c4e47-db27-4251-9c93-8eeff4771941 None None] MFA Rules not processed for user `1754d78f57bd4973ade282eedf52ad2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:26.404579 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-b73c4e47-db27-4251-9c93-8eeff4771941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:26.448475 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:26.499672 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1057/2117] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:26 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:26.504798 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd859cd-3b4f-4966-97c9-a2f276428fa4 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:26.582923 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd859cd-3b4f-4966-97c9-a2f276428fa4 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:26.589199 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083fb19d-0be1-4393-b86a-6950acf03d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:26.589947 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083fb19d-0be1-4393-b86a-6950acf03d37 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:26.590331 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083fb19d-0be1-4393-b86a-6950acf03d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:26.591151 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083fb19d-0be1-4393-b86a-6950acf03d37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:26.591409 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083fb19d-0be1-4393-b86a-6950acf03d37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:26.593484 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083fb19d-0be1-4393-b86a-6950acf03d37 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:26.627734 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1061/2118] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:26 2022] POST /identity/v3/projects => generated 393 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:35:26.631692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c4e47-db27-4251-9c93-8eeff4771941 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:26.643569 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c4e47-db27-4251-9c93-8eeff4771941 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:26.717416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65623f71-c584-4caf-bb4d-b96eb4f5d7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:26.718101 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65623f71-c584-4caf-bb4d-b96eb4f5d7b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:26.718464 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65623f71-c584-4caf-bb4d-b96eb4f5d7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:26.718834 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65623f71-c584-4caf-bb4d-b96eb4f5d7b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:26.719565 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65623f71-c584-4caf-bb4d-b96eb4f5d7b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:26.721545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65623f71-c584-4caf-bb4d-b96eb4f5d7b7 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:26.743969 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1058/2119] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:26 2022] POST /identity/v3/users => generated 303 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:26.747883 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083fb19d-0be1-4393-b86a-6950acf03d37 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:26.758387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083fb19d-0be1-4393-b86a-6950acf03d37 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:26.762918 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5489a-6b08-411b-b73f-71fb18524587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:26.763514 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5489a-6b08-411b-b73f-71fb18524587 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:26.763796 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5489a-6b08-411b-b73f-71fb18524587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:26.764070 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5489a-6b08-411b-b73f-71fb18524587 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:26.764703 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad5489a-6b08-411b-b73f-71fb18524587 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:26.766306 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad5489a-6b08-411b-b73f-71fb18524587 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:26.776845 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1062/2120] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:26.780779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65623f71-c584-4caf-bb4d-b96eb4f5d7b7 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:26.791030 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65623f71-c584-4caf-bb4d-b96eb4f5d7b7 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:26.795493 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cff5b9-116c-4608-b367-b156cfae9f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:26.796137 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cff5b9-116c-4608-b367-b156cfae9f37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:26.796419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cff5b9-116c-4608-b367-b156cfae9f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:26.798108 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cff5b9-116c-4608-b367-b156cfae9f37 admin admin] PATH_INFO: `/v3/projects/8a279f4c65e0437a9ba784e373bfac97/users/39656d43569e402392cf5d4452a11aa9/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:26.815040 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cff5b9-116c-4608-b367-b156cfae9f37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a279f4c65e0437a9ba784e373bfac97, user_id=39656d43569e402392cf5d4452a11aa9, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:26.817748 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cff5b9-116c-4608-b367-b156cfae9f37 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:26.831895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1059/2121] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:26 2022] PUT /identity/v3/projects/8a279f4c65e0437a9ba784e373bfac97/users/39656d43569e402392cf5d4452a11aa9/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:26.836022 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5489a-6b08-411b-b73f-71fb18524587 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:26.845904 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5489a-6b08-411b-b73f-71fb18524587 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:26.908264 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc53895-b637-4cf3-b393-351eb113ca7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:26.909009 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc53895-b637-4cf3-b393-351eb113ca7b admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:26.909295 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc53895-b637-4cf3-b393-351eb113ca7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:26.909574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc53895-b637-4cf3-b393-351eb113ca7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:26.910219 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc53895-b637-4cf3-b393-351eb113ca7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:26.911993 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc53895-b637-4cf3-b393-351eb113ca7b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:26.922603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1063/2122] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:35:26.926645 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cff5b9-116c-4608-b367-b156cfae9f37 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:26.938497 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cff5b9-116c-4608-b367-b156cfae9f37 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:26.944541 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883cc2d-a226-49b0-a1d8-e080d3a06fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:26.945774 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883cc2d-a226-49b0-a1d8-e080d3a06fbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:26.946257 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883cc2d-a226-49b0-a1d8-e080d3a06fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:26.946634 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883cc2d-a226-49b0-a1d8-e080d3a06fbd admin admin] PATH_INFO: `/v3/projects/8a279f4c65e0437a9ba784e373bfac97/users/39656d43569e402392cf5d4452a11aa9/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:26.950035 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5883cc2d-a226-49b0-a1d8-e080d3a06fbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a279f4c65e0437a9ba784e373bfac97, user_id=39656d43569e402392cf5d4452a11aa9, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:26.952702 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5883cc2d-a226-49b0-a1d8-e080d3a06fbd admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:26.962817 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1060/2123] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:26 2022] PUT /identity/v3/projects/8a279f4c65e0437a9ba784e373bfac97/users/39656d43569e402392cf5d4452a11aa9/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:26.972842 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6f45d-4362-4340-a9ec-4c96c497ec7f None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:26.973131 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6f45d-4362-4340-a9ec-4c96c497ec7f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:26.973407 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6f45d-4362-4340-a9ec-4c96c497ec7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:27.002116 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-f1c6f45d-4362-4340-a9ec-4c96c497ec7f None None] MFA Rules not processed for user `39656d43569e402392cf5d4452a11aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:27.003365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f1c6f45d-4362-4340-a9ec-4c96c497ec7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:27.012547 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:27.072618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1064/2124] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:26 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:27.084745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883cc2d-a226-49b0-a1d8-e080d3a06fbd admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:27.086495 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-5883cc2d-a226-49b0-a1d8-e080d3a06fbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:27.144763 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883cc2d-a226-49b0-a1d8-e080d3a06fbd admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:27.148374 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a5280-291d-44dd-a22d-71365d125686 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:27.148979 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a5280-291d-44dd-a22d-71365d125686 service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:27.149762 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a5280-291d-44dd-a22d-71365d125686 service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:27.150056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a5280-291d-44dd-a22d-71365d125686 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:27.160004 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a5280-291d-44dd-a22d-71365d125686 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:27.161810 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a5280-291d-44dd-a22d-71365d125686 service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:27.172615 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1061/2125] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:27 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:27.217856 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6f45d-4362-4340-a9ec-4c96c497ec7f None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:27.227249 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6f45d-4362-4340-a9ec-4c96c497ec7f None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:27.232579 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc7da0-911f-4630-a597-525b49b516bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:27.233294 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc7da0-911f-4630-a597-525b49b516bb service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:27.233577 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc7da0-911f-4630-a597-525b49b516bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:27.233852 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc7da0-911f-4630-a597-525b49b516bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:27.296542 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bc7da0-911f-4630-a597-525b49b516bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:27.298365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bc7da0-911f-4630-a597-525b49b516bb service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:27.307891 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1065/2126] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:27 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:27.958811 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedfb73a-a1cd-4480-a177-71fa3a94922b None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:27.959369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedfb73a-a1cd-4480-a177-71fa3a94922b None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:27.960214 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedfb73a-a1cd-4480-a177-71fa3a94922b None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:27.962867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1062/2127] 10.222.0.18 () {54 vars in 921 bytes} [Thu Sep 1 16:35:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:35:27.978424 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ccbb8-94a0-45ec-a68e-192b6b2e15ce None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:27.979231 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ccbb8-94a0-45ec-a68e-192b6b2e15ce None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:27.979397 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ccbb8-94a0-45ec-a68e-192b6b2e15ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:28.001064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-cb3ccbb8-94a0-45ec-a68e-192b6b2e15ce None None] MFA Rules not processed for user `3ddfb151bbf844efae73ec1990e7eaa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:28.002440 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-cb3ccbb8-94a0-45ec-a68e-192b6b2e15ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:28.088677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1066/2128] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Sep 1 16:35:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:28.100461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedfb73a-a1cd-4480-a177-71fa3a94922b None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:28.103081 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-cedfb73a-a1cd-4480-a177-71fa3a94922b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:28.120535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedfb73a-a1cd-4480-a177-71fa3a94922b None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:28.126263 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3ebb7-e0a4-4740-8c95-007f26d46561 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ddfb151bbf844efae73ec1990e7eaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:28.126963 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3ebb7-e0a4-4740-8c95-007f26d46561 service placement] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:28.127317 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3ebb7-e0a4-4740-8c95-007f26d46561 service placement] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:28.127663 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3ebb7-e0a4-4740-8c95-007f26d46561 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:28.137085 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3ebb7-e0a4-4740-8c95-007f26d46561 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:28.139031 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3ebb7-e0a4-4740-8c95-007f26d46561 service placement] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:28.150286 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1063/2129] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:28.585904 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e09dce-1e0c-46a2-840e-349659fb8a31 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:28.586281 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e09dce-1e0c-46a2-840e-349659fb8a31 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:28.586632 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e09dce-1e0c-46a2-840e-349659fb8a31 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:28.588060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1067/2130] 10.222.0.18 () {54 vars in 921 bytes} [Thu Sep 1 16:35:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:35:28.602734 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75771d73-9ea4-482a-8e76-b8cf7c2eec93 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:28.603016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75771d73-9ea4-482a-8e76-b8cf7c2eec93 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:28.603288 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75771d73-9ea4-482a-8e76-b8cf7c2eec93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:28.623484 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-75771d73-9ea4-482a-8e76-b8cf7c2eec93 None None] MFA Rules not processed for user `3ddfb151bbf844efae73ec1990e7eaa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:28.624781 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-75771d73-9ea4-482a-8e76-b8cf7c2eec93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:28.635377 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1064/2131] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Sep 1 16:35:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:35:28.651990 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e09dce-1e0c-46a2-840e-349659fb8a31 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:28.677954 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75771d73-9ea4-482a-8e76-b8cf7c2eec93 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:28.677954 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-75771d73-9ea4-482a-8e76-b8cf7c2eec93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:28.679937 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e09dce-1e0c-46a2-840e-349659fb8a31 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:28.684757 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:28.685590 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222094a0-27dd-4172-b568-ff051b52836b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ddfb151bbf844efae73ec1990e7eaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:28.686734 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222094a0-27dd-4172-b568-ff051b52836b service placement] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:28.687042 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222094a0-27dd-4172-b568-ff051b52836b service placement] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:28.687312 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222094a0-27dd-4172-b568-ff051b52836b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:28.697214 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222094a0-27dd-4172-b568-ff051b52836b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:28.698863 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222094a0-27dd-4172-b568-ff051b52836b service placement] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:28.708986 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1068/2132] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:28.749449 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75771d73-9ea4-482a-8e76-b8cf7c2eec93 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:28.753998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff863a08-f2e5-4a9f-8f87-0bf129441792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:28.754666 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff863a08-f2e5-4a9f-8f87-0bf129441792 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:28.754950 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff863a08-f2e5-4a9f-8f87-0bf129441792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:28.755228 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff863a08-f2e5-4a9f-8f87-0bf129441792 admin admin] PATH_INFO: `/v3/users/130f67aab4344f7caaac517374e5dabd` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:28.756369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff863a08-f2e5-4a9f-8f87-0bf129441792 admin admin] RBAC: Authorizing `identity:delete_user(user_id=130f67aab4344f7caaac517374e5dabd)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:28.758330 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff863a08-f2e5-4a9f-8f87-0bf129441792 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:28.803496 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-ff863a08-f2e5-4a9f-8f87-0bf129441792 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '130f67aab4344f7caaac517374e5dabd'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:28.804182 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-ff863a08-f2e5-4a9f-8f87-0bf129441792 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '130f67aab4344f7caaac517374e5dabd'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:28.812334 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-ff863a08-f2e5-4a9f-8f87-0bf129441792 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '130f67aab4344f7caaac517374e5dabd'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:28.812983 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-ff863a08-f2e5-4a9f-8f87-0bf129441792 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '130f67aab4344f7caaac517374e5dabd'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:28.825454 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-ff863a08-f2e5-4a9f-8f87-0bf129441792 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '130f67aab4344f7caaac517374e5dabd'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:28.837461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1065/2133] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:28 2022] DELETE /identity/v3/users/130f67aab4344f7caaac517374e5dabd => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:29.011387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222094a0-27dd-4172-b568-ff051b52836b service placement] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:29.024663 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222094a0-27dd-4172-b568-ff051b52836b service placement] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:29.070690 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:29.106616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbd565-f506-4e40-886e-0fd156628632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:29.108882 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbd565-f506-4e40-886e-0fd156628632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:29.109196 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbd565-f506-4e40-886e-0fd156628632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:29.109482 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbd565-f506-4e40-886e-0fd156628632 admin admin] PATH_INFO: `/v3/projects/0cc4a244d7674e6196305091482c90b5` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:29.110677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cbd565-f506-4e40-886e-0fd156628632 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cc4a244d7674e6196305091482c90b5)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:29.112579 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cbd565-f506-4e40-886e-0fd156628632 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:29.161562 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-52cbd565-f506-4e40-886e-0fd156628632 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cc4a244d7674e6196305091482c90b5', 'request_id': 'req-52cbd565-f506-4e40-886e-0fd156628632'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:29.168371 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-52cbd565-f506-4e40-886e-0fd156628632 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cc4a244d7674e6196305091482c90b5', 'request_id': 'req-52cbd565-f506-4e40-886e-0fd156628632'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:29.175925 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-52cbd565-f506-4e40-886e-0fd156628632 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cc4a244d7674e6196305091482c90b5', 'request_id': 'req-52cbd565-f506-4e40-886e-0fd156628632'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:29.187526 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-52cbd565-f506-4e40-886e-0fd156628632 admin admin] The token cache is being invalidate because project 0cc4a244d7674e6196305091482c90b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:29.187816 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-52cbd565-f506-4e40-886e-0fd156628632 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:29.188435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-52cbd565-f506-4e40-886e-0fd156628632 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:29.189904 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1069/2134] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:29 2022] DELETE /identity/v3/projects/0cc4a244d7674e6196305091482c90b5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:29.194298 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff863a08-f2e5-4a9f-8f87-0bf129441792 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:29.195721 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-ff863a08-f2e5-4a9f-8f87-0bf129441792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:29.209019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff863a08-f2e5-4a9f-8f87-0bf129441792 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:29.265471 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43478f-b7e6-499a-a3eb-0a4ada798ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:29.266103 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43478f-b7e6-499a-a3eb-0a4ada798ebc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:29.266424 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43478f-b7e6-499a-a3eb-0a4ada798ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:29.266698 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43478f-b7e6-499a-a3eb-0a4ada798ebc admin admin] PATH_INFO: `/v3/users/6a85dce071b140938855d264334c491b` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:29.267806 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e43478f-b7e6-499a-a3eb-0a4ada798ebc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a85dce071b140938855d264334c491b)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:29.269734 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e43478f-b7e6-499a-a3eb-0a4ada798ebc admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:29.300850 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:29.330907 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-7e43478f-b7e6-499a-a3eb-0a4ada798ebc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a85dce071b140938855d264334c491b'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:29.331761 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-7e43478f-b7e6-499a-a3eb-0a4ada798ebc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a85dce071b140938855d264334c491b'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:29.342397 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-7e43478f-b7e6-499a-a3eb-0a4ada798ebc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a85dce071b140938855d264334c491b'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:29.343263 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-7e43478f-b7e6-499a-a3eb-0a4ada798ebc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a85dce071b140938855d264334c491b'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:29.372672 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-7e43478f-b7e6-499a-a3eb-0a4ada798ebc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a85dce071b140938855d264334c491b'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:29.389325 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1066/2135] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:29 2022] DELETE /identity/v3/users/6a85dce071b140938855d264334c491b => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:29.563230 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbd565-f506-4e40-886e-0fd156628632 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:29.564684 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-52cbd565-f506-4e40-886e-0fd156628632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:29.579058 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbd565-f506-4e40-886e-0fd156628632 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:29.643878 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42054919-8afa-4762-b432-b96b11fcbf75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:29.644448 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42054919-8afa-4762-b432-b96b11fcbf75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:29.644731 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42054919-8afa-4762-b432-b96b11fcbf75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:29.645007 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42054919-8afa-4762-b432-b96b11fcbf75 admin admin] PATH_INFO: `/v3/projects/5cf49b872ceb410c84471659ddfb2291` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:29.646119 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42054919-8afa-4762-b432-b96b11fcbf75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cf49b872ceb410c84471659ddfb2291)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:29.648080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42054919-8afa-4762-b432-b96b11fcbf75 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:29.700957 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-42054919-8afa-4762-b432-b96b11fcbf75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cf49b872ceb410c84471659ddfb2291', 'request_id': 'req-42054919-8afa-4762-b432-b96b11fcbf75'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:29.707577 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-42054919-8afa-4762-b432-b96b11fcbf75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cf49b872ceb410c84471659ddfb2291', 'request_id': 'req-42054919-8afa-4762-b432-b96b11fcbf75'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:29.723497 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-42054919-8afa-4762-b432-b96b11fcbf75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cf49b872ceb410c84471659ddfb2291', 'request_id': 'req-42054919-8afa-4762-b432-b96b11fcbf75'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:29.725085 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:29.739461 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-42054919-8afa-4762-b432-b96b11fcbf75 admin admin] The token cache is being invalidate because project 5cf49b872ceb410c84471659ddfb2291 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:29.739776 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-42054919-8afa-4762-b432-b96b11fcbf75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:29.740786 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-42054919-8afa-4762-b432-b96b11fcbf75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:29.742542 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1070/2136] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:29 2022] DELETE /identity/v3/projects/5cf49b872ceb410c84471659ddfb2291 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:29.747696 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d0025-d175-4098-9902-c790f9ca0bad None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:29.748011 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d0025-d175-4098-9902-c790f9ca0bad None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:29.748288 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d0025-d175-4098-9902-c790f9ca0bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:29.768692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-a82d0025-d175-4098-9902-c790f9ca0bad None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:29.769886 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-a82d0025-d175-4098-9902-c790f9ca0bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:29.829707 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1067/2137] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:29.837974 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c626ef-c217-40bb-b5fe-ddd68b3e9675 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:29.838317 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c626ef-c217-40bb-b5fe-ddd68b3e9675 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:29.838600 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c626ef-c217-40bb-b5fe-ddd68b3e9675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:29.858734 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-b7c626ef-c217-40bb-b5fe-ddd68b3e9675 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:29.859939 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b7c626ef-c217-40bb-b5fe-ddd68b3e9675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:29.867711 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1071/2138] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:29.871898 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d0025-d175-4098-9902-c790f9ca0bad None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:29.881507 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d0025-d175-4098-9902-c790f9ca0bad None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:29.886528 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fee739-f632-4367-aa57-9a8fc058da26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:29.887123 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fee739-f632-4367-aa57-9a8fc058da26 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:29.887405 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fee739-f632-4367-aa57-9a8fc058da26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:29.887678 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fee739-f632-4367-aa57-9a8fc058da26 admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:29.888320 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fee739-f632-4367-aa57-9a8fc058da26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:35:29.888624 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fee739-f632-4367-aa57-9a8fc058da26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:29.890217 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fee739-f632-4367-aa57-9a8fc058da26 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:29.902457 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1068/2139] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:35:29.906584 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c626ef-c217-40bb-b5fe-ddd68b3e9675 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:29.915792 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c626ef-c217-40bb-b5fe-ddd68b3e9675 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:29.920392 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9cdfc-bcfb-4f68-8958-01a5f7ed5d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:29.921001 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9cdfc-bcfb-4f68-8958-01a5f7ed5d0a admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:29.921279 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9cdfc-bcfb-4f68-8958-01a5f7ed5d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:29.921550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9cdfc-bcfb-4f68-8958-01a5f7ed5d0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:29.922176 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f9cdfc-bcfb-4f68-8958-01a5f7ed5d0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:29.924095 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f9cdfc-bcfb-4f68-8958-01a5f7ed5d0a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:29.950888 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1072/2140] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:29 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:29.955173 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fee739-f632-4367-aa57-9a8fc058da26 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:29.967516 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fee739-f632-4367-aa57-9a8fc058da26 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:30.033423 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ddf74-c020-4b1b-92cd-c52854b861af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:30.034353 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ddf74-c020-4b1b-92cd-c52854b861af admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:30.034644 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ddf74-c020-4b1b-92cd-c52854b861af admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:30.035016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ddf74-c020-4b1b-92cd-c52854b861af admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:30.035723 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0ddf74-c020-4b1b-92cd-c52854b861af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:30.038034 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0ddf74-c020-4b1b-92cd-c52854b861af admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:30.061912 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1069/2141] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:29 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:30.066519 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9cdfc-bcfb-4f68-8958-01a5f7ed5d0a admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:30.076870 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9cdfc-bcfb-4f68-8958-01a5f7ed5d0a admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:30.081570 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644d700-81b8-4155-9b30-cf5481b4ef68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:30.082175 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644d700-81b8-4155-9b30-cf5481b4ef68 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:30.082504 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644d700-81b8-4155-9b30-cf5481b4ef68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:30.082781 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644d700-81b8-4155-9b30-cf5481b4ef68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:30.083427 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c644d700-81b8-4155-9b30-cf5481b4ef68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:30.085150 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c644d700-81b8-4155-9b30-cf5481b4ef68 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:30.096367 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1073/2142] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:35:30.101282 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ddf74-c020-4b1b-92cd-c52854b861af admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:30.120131 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ddf74-c020-4b1b-92cd-c52854b861af admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:30.133209 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e5ee3-8d7c-43d4-a820-7a6b763b528d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:30.133954 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e5ee3-8d7c-43d4-a820-7a6b763b528d admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:30.135064 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e5ee3-8d7c-43d4-a820-7a6b763b528d admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:30.135535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e5ee3-8d7c-43d4-a820-7a6b763b528d admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:30.139686 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e5ee3-8d7c-43d4-a820-7a6b763b528d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:30.141395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e5ee3-8d7c-43d4-a820-7a6b763b528d admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:30.162367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1070/2143] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:30.165876 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644d700-81b8-4155-9b30-cf5481b4ef68 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:30.177683 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644d700-81b8-4155-9b30-cf5481b4ef68 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:30.188491 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6feb0aa-dd00-4acf-b010-c1c59e7197c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:30.189165 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6feb0aa-dd00-4acf-b010-c1c59e7197c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:30.189454 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6feb0aa-dd00-4acf-b010-c1c59e7197c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:30.189734 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6feb0aa-dd00-4acf-b010-c1c59e7197c7 admin admin] PATH_INFO: `/v3/projects/e0142c39dbb14868b0826834b575d539/users/33b31c49ceaf4108b41d6b5ad12cd98c/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:30.213790 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6feb0aa-dd00-4acf-b010-c1c59e7197c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0142c39dbb14868b0826834b575d539, user_id=33b31c49ceaf4108b41d6b5ad12cd98c, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:30.217443 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6feb0aa-dd00-4acf-b010-c1c59e7197c7 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:30.233857 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1074/2144] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:30 2022] PUT /identity/v3/projects/e0142c39dbb14868b0826834b575d539/users/33b31c49ceaf4108b41d6b5ad12cd98c/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:30.245269 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561e339-efe0-4879-b471-1b1bb1bfacb8 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:30.245805 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561e339-efe0-4879-b471-1b1bb1bfacb8 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:30.245805 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561e339-efe0-4879-b471-1b1bb1bfacb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:30.267635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:30.279853 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-8561e339-efe0-4879-b471-1b1bb1bfacb8 None None] MFA Rules not processed for user `33b31c49ceaf4108b41d6b5ad12cd98c`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:30.282078 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-8561e339-efe0-4879-b471-1b1bb1bfacb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:30.381179 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1071/2145] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:30.390054 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6feb0aa-dd00-4acf-b010-c1c59e7197c7 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:30.392029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d6feb0aa-dd00-4acf-b010-c1c59e7197c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:30.458634 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6feb0aa-dd00-4acf-b010-c1c59e7197c7 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:30.463128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931bbc3-730d-408a-b8e1-f11bfe7810d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:30.463728 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931bbc3-730d-408a-b8e1-f11bfe7810d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:30.464010 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931bbc3-730d-408a-b8e1-f11bfe7810d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:30.464287 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931bbc3-730d-408a-b8e1-f11bfe7810d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:30.474497 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931bbc3-730d-408a-b8e1-f11bfe7810d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:30.475934 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931bbc3-730d-408a-b8e1-f11bfe7810d1 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:30.485409 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1075/2146] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:35:34.892164 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561e339-efe0-4879-b471-1b1bb1bfacb8 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:34.893826 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-8561e339-efe0-4879-b471-1b1bb1bfacb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:34.995428 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561e339-efe0-4879-b471-1b1bb1bfacb8 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:35.003277 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7f190-43eb-49bc-adec-694ff3648ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:35.003941 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7f190-43eb-49bc-adec-694ff3648ad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:35.004230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7f190-43eb-49bc-adec-694ff3648ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:35.004506 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7f190-43eb-49bc-adec-694ff3648ad1 admin admin] PATH_INFO: `/v3/users/1754d78f57bd4973ade282eedf52ad2d` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:35.005667 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad7f190-43eb-49bc-adec-694ff3648ad1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1754d78f57bd4973ade282eedf52ad2d)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:35.008742 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad7f190-43eb-49bc-adec-694ff3648ad1 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:35.087882 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5ad7f190-43eb-49bc-adec-694ff3648ad1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1754d78f57bd4973ade282eedf52ad2d'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:35.088731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5ad7f190-43eb-49bc-adec-694ff3648ad1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1754d78f57bd4973ade282eedf52ad2d'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:35.105456 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5ad7f190-43eb-49bc-adec-694ff3648ad1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1754d78f57bd4973ade282eedf52ad2d'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:35.108122 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5ad7f190-43eb-49bc-adec-694ff3648ad1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1754d78f57bd4973ade282eedf52ad2d'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:35.134363 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5ad7f190-43eb-49bc-adec-694ff3648ad1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1754d78f57bd4973ade282eedf52ad2d'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:35.165127 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1072/2147] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:34 2022] DELETE /identity/v3/users/1754d78f57bd4973ade282eedf52ad2d => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:35.166758 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2ae2f-ca95-47bf-8e83-7fac7e17c330 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:35.167257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2ae2f-ca95-47bf-8e83-7fac7e17c330 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:35.167332 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2ae2f-ca95-47bf-8e83-7fac7e17c330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:35.197463 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-79b2ae2f-ca95-47bf-8e83-7fac7e17c330 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:35.204132 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-79b2ae2f-ca95-47bf-8e83-7fac7e17c330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:35.286673 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1076/2148] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:35.298485 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7f190-43eb-49bc-adec-694ff3648ad1 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:35.303511 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-5ad7f190-43eb-49bc-adec-694ff3648ad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:35.336430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:35.383232 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7f190-43eb-49bc-adec-694ff3648ad1 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:35.386902 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0dffd-d5bb-4bef-a5e7-7076dc485100 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:35.387617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0dffd-d5bb-4bef-a5e7-7076dc485100 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:35.387936 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0dffd-d5bb-4bef-a5e7-7076dc485100 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:35.388213 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0dffd-d5bb-4bef-a5e7-7076dc485100 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:35.398310 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce0dffd-d5bb-4bef-a5e7-7076dc485100 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:35.400455 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce0dffd-d5bb-4bef-a5e7-7076dc485100 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:35.419404 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1073/2149] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:35.713513 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2ae2f-ca95-47bf-8e83-7fac7e17c330 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:35.715108 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-79b2ae2f-ca95-47bf-8e83-7fac7e17c330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:35.731431 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2ae2f-ca95-47bf-8e83-7fac7e17c330 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:35.747710 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35dcbda-5b74-436f-87f9-879a8653ca11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:35.748766 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dcbda-5b74-436f-87f9-879a8653ca11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:35.749272 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dcbda-5b74-436f-87f9-879a8653ca11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:35.750204 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dcbda-5b74-436f-87f9-879a8653ca11 admin admin] PATH_INFO: `/v3/projects/908c30c0feaf418ab8b991eb441ba2e8` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:35.752730 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35dcbda-5b74-436f-87f9-879a8653ca11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=908c30c0feaf418ab8b991eb441ba2e8)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:35.754734 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35dcbda-5b74-436f-87f9-879a8653ca11 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:35.784910 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:35.814911 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d35dcbda-5b74-436f-87f9-879a8653ca11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '908c30c0feaf418ab8b991eb441ba2e8', 'request_id': 'req-d35dcbda-5b74-436f-87f9-879a8653ca11'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:35.821875 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d35dcbda-5b74-436f-87f9-879a8653ca11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '908c30c0feaf418ab8b991eb441ba2e8', 'request_id': 'req-d35dcbda-5b74-436f-87f9-879a8653ca11'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:35.829491 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d35dcbda-5b74-436f-87f9-879a8653ca11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '908c30c0feaf418ab8b991eb441ba2e8', 'request_id': 'req-d35dcbda-5b74-436f-87f9-879a8653ca11'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:35.841733 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d35dcbda-5b74-436f-87f9-879a8653ca11 admin admin] The token cache is being invalidate because project 908c30c0feaf418ab8b991eb441ba2e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:35.842030 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d35dcbda-5b74-436f-87f9-879a8653ca11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:35.842728 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d35dcbda-5b74-436f-87f9-879a8653ca11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:35.844446 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1077/2150] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:35 2022] DELETE /identity/v3/projects/908c30c0feaf418ab8b991eb441ba2e8 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:35.848221 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0dffd-d5bb-4bef-a5e7-7076dc485100 service neutron] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:35.859032 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0dffd-d5bb-4bef-a5e7-7076dc485100 service neutron] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:35.915566 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53faf3-4e2b-40c9-9db4-598a8628a9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:35.916274 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53faf3-4e2b-40c9-9db4-598a8628a9b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:35.916471 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53faf3-4e2b-40c9-9db4-598a8628a9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:35.916748 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53faf3-4e2b-40c9-9db4-598a8628a9b6 admin admin] PATH_INFO: `/v3/users/39656d43569e402392cf5d4452a11aa9` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:35.917881 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb53faf3-4e2b-40c9-9db4-598a8628a9b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39656d43569e402392cf5d4452a11aa9)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:35.919902 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb53faf3-4e2b-40c9-9db4-598a8628a9b6 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:35.970694 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-fb53faf3-4e2b-40c9-9db4-598a8628a9b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39656d43569e402392cf5d4452a11aa9'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:35.973674 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-fb53faf3-4e2b-40c9-9db4-598a8628a9b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39656d43569e402392cf5d4452a11aa9'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:35.983490 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-fb53faf3-4e2b-40c9-9db4-598a8628a9b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39656d43569e402392cf5d4452a11aa9'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:35.985669 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-fb53faf3-4e2b-40c9-9db4-598a8628a9b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39656d43569e402392cf5d4452a11aa9'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:36.001368 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-fb53faf3-4e2b-40c9-9db4-598a8628a9b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39656d43569e402392cf5d4452a11aa9'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:36.020175 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1074/2151] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:35 2022] DELETE /identity/v3/users/39656d43569e402392cf5d4452a11aa9 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:36.304301 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3620b-cf1c-4b6f-9ead-91000aee2b28 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:36.305297 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3620b-cf1c-4b6f-9ead-91000aee2b28 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:36.305898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3620b-cf1c-4b6f-9ead-91000aee2b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:36.324614 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53faf3-4e2b-40c9-9db4-598a8628a9b6 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:36.326470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-fb53faf3-4e2b-40c9-9db4-598a8628a9b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:36.367589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-cba3620b-cf1c-4b6f-9ead-91000aee2b28 None None] MFA Rules not processed for user `7425d9aefcf940709f3a3aad4166755c`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:36.369260 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-cba3620b-cf1c-4b6f-9ead-91000aee2b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:36.415799 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:36.441778 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53faf3-4e2b-40c9-9db4-598a8628a9b6 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:36.443373 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1078/2152] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:36.448512 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5508846e-63fa-4749-98fe-c761ed531feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:36.449155 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508846e-63fa-4749-98fe-c761ed531feb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:36.449434 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508846e-63fa-4749-98fe-c761ed531feb admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:36.449709 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508846e-63fa-4749-98fe-c761ed531feb admin admin] PATH_INFO: `/v3/projects/8a279f4c65e0437a9ba784e373bfac97` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:36.450846 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508846e-63fa-4749-98fe-c761ed531feb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a279f4c65e0437a9ba784e373bfac97)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:36.452760 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508846e-63fa-4749-98fe-c761ed531feb admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:36.519943 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5508846e-63fa-4749-98fe-c761ed531feb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a279f4c65e0437a9ba784e373bfac97', 'request_id': 'req-5508846e-63fa-4749-98fe-c761ed531feb'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:36.529122 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5508846e-63fa-4749-98fe-c761ed531feb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a279f4c65e0437a9ba784e373bfac97', 'request_id': 'req-5508846e-63fa-4749-98fe-c761ed531feb'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:36.536653 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5508846e-63fa-4749-98fe-c761ed531feb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a279f4c65e0437a9ba784e373bfac97', 'request_id': 'req-5508846e-63fa-4749-98fe-c761ed531feb'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:36.549982 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5508846e-63fa-4749-98fe-c761ed531feb admin admin] The token cache is being invalidate because project 8a279f4c65e0437a9ba784e373bfac97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:36.550444 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5508846e-63fa-4749-98fe-c761ed531feb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:36.551160 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5508846e-63fa-4749-98fe-c761ed531feb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:36.556067 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1075/2153] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:36 2022] DELETE /identity/v3/projects/8a279f4c65e0437a9ba784e373bfac97 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:36.557629 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a2fee-da63-4f6c-9b40-3683a9783e71 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:36.557917 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a2fee-da63-4f6c-9b40-3683a9783e71 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:36.560648 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a2fee-da63-4f6c-9b40-3683a9783e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:36.589171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-c33a2fee-da63-4f6c-9b40-3683a9783e71 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:36.595535 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c33a2fee-da63-4f6c-9b40-3683a9783e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:36.699750 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1079/2154] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:35:36.707123 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f43ce4-6b44-4a5c-9662-075fbc1f42fa None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:36.707687 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f43ce4-6b44-4a5c-9662-075fbc1f42fa None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:36.708153 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f43ce4-6b44-4a5c-9662-075fbc1f42fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:36.732160 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-a3f43ce4-6b44-4a5c-9662-075fbc1f42fa None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:36.733398 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-a3f43ce4-6b44-4a5c-9662-075fbc1f42fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:36.742849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1076/2155] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:36.746740 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a2fee-da63-4f6c-9b40-3683a9783e71 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:36.757270 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a2fee-da63-4f6c-9b40-3683a9783e71 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:36.763027 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7d192-6a82-4b01-98fb-bc281c50de3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:36.763651 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7d192-6a82-4b01-98fb-bc281c50de3f admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:36.763935 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7d192-6a82-4b01-98fb-bc281c50de3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:36.764211 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7d192-6a82-4b01-98fb-bc281c50de3f admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:36.765149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7d192-6a82-4b01-98fb-bc281c50de3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:35:36.765149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7d192-6a82-4b01-98fb-bc281c50de3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:36.766819 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7d192-6a82-4b01-98fb-bc281c50de3f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:36.785688 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f43ce4-6b44-4a5c-9662-075fbc1f42fa None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:36.795977 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f43ce4-6b44-4a5c-9662-075fbc1f42fa None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:36.796955 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1080/2156] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:35:36.801274 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edddca2-6135-48a3-8cef-b8908ca3102f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:36.801890 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edddca2-6135-48a3-8cef-b8908ca3102f admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:36.802275 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edddca2-6135-48a3-8cef-b8908ca3102f admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:36.802557 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edddca2-6135-48a3-8cef-b8908ca3102f admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:36.803197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edddca2-6135-48a3-8cef-b8908ca3102f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:36.805107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edddca2-6135-48a3-8cef-b8908ca3102f admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:36.837780 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1077/2157] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:36 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:36.841571 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7d192-6a82-4b01-98fb-bc281c50de3f admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:36.852806 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7d192-6a82-4b01-98fb-bc281c50de3f admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:36.899666 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:36.918036 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a272334-e15a-4043-908b-9ba9977613b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:36.919810 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a272334-e15a-4043-908b-9ba9977613b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:36.920247 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a272334-e15a-4043-908b-9ba9977613b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:36.920705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a272334-e15a-4043-908b-9ba9977613b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:36.921491 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a272334-e15a-4043-908b-9ba9977613b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:36.925290 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a272334-e15a-4043-908b-9ba9977613b4 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:36.951414 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1081/2158] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:36 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:35:36.955617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edddca2-6135-48a3-8cef-b8908ca3102f admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:36.966661 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edddca2-6135-48a3-8cef-b8908ca3102f admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:36.971412 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92879a-093a-4c42-9271-e9385fc1fa7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:36.972099 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92879a-093a-4c42-9271-e9385fc1fa7e admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:36.972482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92879a-093a-4c42-9271-e9385fc1fa7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:36.972844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92879a-093a-4c42-9271-e9385fc1fa7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:36.973643 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b92879a-093a-4c42-9271-e9385fc1fa7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:36.975475 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b92879a-093a-4c42-9271-e9385fc1fa7e admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:36.986698 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1078/2159] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:36.990824 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a272334-e15a-4043-908b-9ba9977613b4 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:37.001907 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a272334-e15a-4043-908b-9ba9977613b4 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:37.010182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611d751c-5792-4832-a7db-76a2d3651c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:37.010837 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d751c-5792-4832-a7db-76a2d3651c63 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:37.011854 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d751c-5792-4832-a7db-76a2d3651c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:37.012132 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d751c-5792-4832-a7db-76a2d3651c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:37.012827 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611d751c-5792-4832-a7db-76a2d3651c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:37.014805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611d751c-5792-4832-a7db-76a2d3651c63 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:37.026433 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1082/2160] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:35:37.031619 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92879a-093a-4c42-9271-e9385fc1fa7e admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:37.046144 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92879a-093a-4c42-9271-e9385fc1fa7e admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:37.052365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07566242-9d80-4fd0-92c0-5e70765e5930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:37.053372 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07566242-9d80-4fd0-92c0-5e70765e5930 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:37.053668 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07566242-9d80-4fd0-92c0-5e70765e5930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:37.053948 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07566242-9d80-4fd0-92c0-5e70765e5930 admin admin] PATH_INFO: `/v3/projects/e6395f60c0df449d8b2afe28d84a4851/users/b3f9e14f3b604a9bbba0f75620f28555/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:37.075872 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07566242-9d80-4fd0-92c0-5e70765e5930 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6395f60c0df449d8b2afe28d84a4851, user_id=b3f9e14f3b604a9bbba0f75620f28555, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:37.089288 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07566242-9d80-4fd0-92c0-5e70765e5930 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:37.097993 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1079/2161] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:37 2022] PUT /identity/v3/projects/e6395f60c0df449d8b2afe28d84a4851/users/b3f9e14f3b604a9bbba0f75620f28555/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:37.110832 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40cab7-ae1c-4f36-857c-5f00e296ee34 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:37.111136 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40cab7-ae1c-4f36-857c-5f00e296ee34 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:37.111747 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40cab7-ae1c-4f36-857c-5f00e296ee34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:37.157730 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-3c40cab7-ae1c-4f36-857c-5f00e296ee34 None None] MFA Rules not processed for user `b3f9e14f3b604a9bbba0f75620f28555`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:37.158992 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3c40cab7-ae1c-4f36-857c-5f00e296ee34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:37.265902 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1083/2162] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:37.270069 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07566242-9d80-4fd0-92c0-5e70765e5930 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:37.362354 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07566242-9d80-4fd0-92c0-5e70765e5930 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:37.387494 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f4252-0484-4849-90fa-d4d434b61a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:37.389267 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f4252-0484-4849-90fa-d4d434b61a14 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:37.389676 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f4252-0484-4849-90fa-d4d434b61a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:37.390053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f4252-0484-4849-90fa-d4d434b61a14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:37.390900 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f4252-0484-4849-90fa-d4d434b61a14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:37.393457 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f4252-0484-4849-90fa-d4d434b61a14 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:37.410993 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:37.447142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1080/2163] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:37 2022] POST /identity/v3/projects => generated 385 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:37.450871 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40cab7-ae1c-4f36-857c-5f00e296ee34 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:37.467511 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40cab7-ae1c-4f36-857c-5f00e296ee34 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:37.598552 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a06c25-bb51-4cd2-bbe8-b2f360754e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:37.599260 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a06c25-bb51-4cd2-bbe8-b2f360754e1e admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:37.599469 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a06c25-bb51-4cd2-bbe8-b2f360754e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:37.600042 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a06c25-bb51-4cd2-bbe8-b2f360754e1e admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:37.600393 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a06c25-bb51-4cd2-bbe8-b2f360754e1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:37.602416 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a06c25-bb51-4cd2-bbe8-b2f360754e1e admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:37.645674 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1084/2164] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:37 2022] POST /identity/v3/users => generated 299 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:35:37.649323 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f4252-0484-4849-90fa-d4d434b61a14 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:37.665936 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f4252-0484-4849-90fa-d4d434b61a14 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:37.679976 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b22ae-3aa7-469d-a919-6088df194ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:37.679976 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b22ae-3aa7-469d-a919-6088df194ed3 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:37.679976 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b22ae-3aa7-469d-a919-6088df194ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:37.679976 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b22ae-3aa7-469d-a919-6088df194ed3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:37.686385 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383b22ae-3aa7-469d-a919-6088df194ed3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:37.688565 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383b22ae-3aa7-469d-a919-6088df194ed3 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:37.707007 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1081/2165] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:37.708303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a06c25-bb51-4cd2-bbe8-b2f360754e1e admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:37.722535 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a06c25-bb51-4cd2-bbe8-b2f360754e1e admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:37.730325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eba3cf-53bd-403e-a4bc-76f6cdbd2d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:37.731517 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eba3cf-53bd-403e-a4bc-76f6cdbd2d65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:37.732023 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eba3cf-53bd-403e-a4bc-76f6cdbd2d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:37.732506 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eba3cf-53bd-403e-a4bc-76f6cdbd2d65 admin admin] PATH_INFO: `/v3/projects/372b9ecc883f4bd4a02e544814cb800b/users/d098c0dee61b450780beb848ce176958/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:37.749491 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eba3cf-53bd-403e-a4bc-76f6cdbd2d65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=372b9ecc883f4bd4a02e544814cb800b, user_id=d098c0dee61b450780beb848ce176958, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:37.752327 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eba3cf-53bd-403e-a4bc-76f6cdbd2d65 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:37.773034 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1085/2166] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:37 2022] PUT /identity/v3/projects/372b9ecc883f4bd4a02e544814cb800b/users/d098c0dee61b450780beb848ce176958/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:37.774258 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b22ae-3aa7-469d-a919-6088df194ed3 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:37.785942 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b22ae-3aa7-469d-a919-6088df194ed3 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:37.857210 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082e78a-8e65-4f77-8b86-73f49f5ce204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:37.857569 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082e78a-8e65-4f77-8b86-73f49f5ce204 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:37.858157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082e78a-8e65-4f77-8b86-73f49f5ce204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:37.858157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082e78a-8e65-4f77-8b86-73f49f5ce204 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:37.858786 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4082e78a-8e65-4f77-8b86-73f49f5ce204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:37.860395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4082e78a-8e65-4f77-8b86-73f49f5ce204 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:37.871577 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1082/2167] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:37.876728 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eba3cf-53bd-403e-a4bc-76f6cdbd2d65 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:37.887938 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eba3cf-53bd-403e-a4bc-76f6cdbd2d65 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:37.892531 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102fb568-e1cb-4e34-97e5-5e0a293b0fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:37.893191 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fb568-e1cb-4e34-97e5-5e0a293b0fb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:37.893472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fb568-e1cb-4e34-97e5-5e0a293b0fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:37.893745 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fb568-e1cb-4e34-97e5-5e0a293b0fb4 admin admin] PATH_INFO: `/v3/projects/372b9ecc883f4bd4a02e544814cb800b/users/d098c0dee61b450780beb848ce176958/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:37.895692 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102fb568-e1cb-4e34-97e5-5e0a293b0fb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=372b9ecc883f4bd4a02e544814cb800b, user_id=d098c0dee61b450780beb848ce176958, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:37.898340 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102fb568-e1cb-4e34-97e5-5e0a293b0fb4 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:37.909626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1086/2168] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:37 2022] PUT /identity/v3/projects/372b9ecc883f4bd4a02e544814cb800b/users/d098c0dee61b450780beb848ce176958/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:37.920419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639e468-d104-49a7-a519-df90fb5465be None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:37.920734 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639e468-d104-49a7-a519-df90fb5465be None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:37.921013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639e468-d104-49a7-a519-df90fb5465be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:37.952788 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-4639e468-d104-49a7-a519-df90fb5465be None None] MFA Rules not processed for user `d098c0dee61b450780beb848ce176958`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:37.954027 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-4639e468-d104-49a7-a519-df90fb5465be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:38.025934 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1083/2169] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:37 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:38.036548 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102fb568-e1cb-4e34-97e5-5e0a293b0fb4 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:38.038357 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-102fb568-e1cb-4e34-97e5-5e0a293b0fb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:38.046765 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:38.101690 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102fb568-e1cb-4e34-97e5-5e0a293b0fb4 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:38.106080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66adebd6-5226-4a99-8eb7-6175b066a806 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:38.106743 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adebd6-5226-4a99-8eb7-6175b066a806 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:38.107027 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adebd6-5226-4a99-8eb7-6175b066a806 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:38.107344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adebd6-5226-4a99-8eb7-6175b066a806 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:38.169590 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66adebd6-5226-4a99-8eb7-6175b066a806 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:38.171055 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66adebd6-5226-4a99-8eb7-6175b066a806 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:38.179818 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1087/2170] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:39.448179 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639e468-d104-49a7-a519-df90fb5465be None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:39.450059 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-4639e468-d104-49a7-a519-df90fb5465be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:39.633031 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639e468-d104-49a7-a519-df90fb5465be None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:39.639076 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9256ee-a2d3-4d2b-b3ef-c062f5fbd9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:39.640031 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9256ee-a2d3-4d2b-b3ef-c062f5fbd9f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:39.640458 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9256ee-a2d3-4d2b-b3ef-c062f5fbd9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:39.641033 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9256ee-a2d3-4d2b-b3ef-c062f5fbd9f0 admin admin] PATH_INFO: `/v3/users/b172f56cbd604342ba6d8169c969e157` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:39.643202 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9256ee-a2d3-4d2b-b3ef-c062f5fbd9f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b172f56cbd604342ba6d8169c969e157)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:39.645319 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9256ee-a2d3-4d2b-b3ef-c062f5fbd9f0 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:39.650001 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:39.737915 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-ed9256ee-a2d3-4d2b-b3ef-c062f5fbd9f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b172f56cbd604342ba6d8169c969e157'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:39.739142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-ed9256ee-a2d3-4d2b-b3ef-c062f5fbd9f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b172f56cbd604342ba6d8169c969e157'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:39.757442 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-ed9256ee-a2d3-4d2b-b3ef-c062f5fbd9f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b172f56cbd604342ba6d8169c969e157'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:39.758062 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-ed9256ee-a2d3-4d2b-b3ef-c062f5fbd9f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b172f56cbd604342ba6d8169c969e157'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:39.787252 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-ed9256ee-a2d3-4d2b-b3ef-c062f5fbd9f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b172f56cbd604342ba6d8169c969e157'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:39.813297 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1084/2171] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:39 2022] DELETE /identity/v3/users/b172f56cbd604342ba6d8169c969e157 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:40.048156 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66adebd6-5226-4a99-8eb7-6175b066a806 service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:40.062994 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66adebd6-5226-4a99-8eb7-6175b066a806 service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:40.146965 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38443049-9ff3-4bee-9409-823d5d97511d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:40.147737 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38443049-9ff3-4bee-9409-823d5d97511d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:40.148238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38443049-9ff3-4bee-9409-823d5d97511d admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:40.149087 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38443049-9ff3-4bee-9409-823d5d97511d admin admin] PATH_INFO: `/v3/projects/5c40a340919a49cc8b73687df600e2ea` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:40.150489 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38443049-9ff3-4bee-9409-823d5d97511d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c40a340919a49cc8b73687df600e2ea)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:40.152619 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38443049-9ff3-4bee-9409-823d5d97511d admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:40.244693 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-38443049-9ff3-4bee-9409-823d5d97511d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c40a340919a49cc8b73687df600e2ea', 'request_id': 'req-38443049-9ff3-4bee-9409-823d5d97511d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:40.255898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-38443049-9ff3-4bee-9409-823d5d97511d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c40a340919a49cc8b73687df600e2ea', 'request_id': 'req-38443049-9ff3-4bee-9409-823d5d97511d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:40.267569 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-38443049-9ff3-4bee-9409-823d5d97511d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c40a340919a49cc8b73687df600e2ea', 'request_id': 'req-38443049-9ff3-4bee-9409-823d5d97511d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:40.269765 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:40.287777 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-38443049-9ff3-4bee-9409-823d5d97511d admin admin] The token cache is being invalidate because project 5c40a340919a49cc8b73687df600e2ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:40.288305 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-38443049-9ff3-4bee-9409-823d5d97511d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:40.290515 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-38443049-9ff3-4bee-9409-823d5d97511d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:40.292844 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1088/2172] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:40 2022] DELETE /identity/v3/projects/5c40a340919a49cc8b73687df600e2ea => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:40.303249 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695eb21-067e-4c92-b98e-67dbec73b6e3 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:40.303540 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695eb21-067e-4c92-b98e-67dbec73b6e3 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:40.303921 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695eb21-067e-4c92-b98e-67dbec73b6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:40.351463 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-5695eb21-067e-4c92-b98e-67dbec73b6e3 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:40.353949 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-5695eb21-067e-4c92-b98e-67dbec73b6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:40.476312 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1085/2173] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:40.484951 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960196c1-6c20-41c2-b6c7-4c9a5a873dd9 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:40.485238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960196c1-6c20-41c2-b6c7-4c9a5a873dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:40.485514 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960196c1-6c20-41c2-b6c7-4c9a5a873dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:40.506867 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-960196c1-6c20-41c2-b6c7-4c9a5a873dd9 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:40.508092 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-960196c1-6c20-41c2-b6c7-4c9a5a873dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:40.522531 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1089/2174] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:40.526856 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695eb21-067e-4c92-b98e-67dbec73b6e3 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:40.538308 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695eb21-067e-4c92-b98e-67dbec73b6e3 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:40.543380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d87f0ca-a0f5-406b-a975-8f57c94083cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:40.543982 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87f0ca-a0f5-406b-a975-8f57c94083cc admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:40.544261 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87f0ca-a0f5-406b-a975-8f57c94083cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:40.544532 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87f0ca-a0f5-406b-a975-8f57c94083cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:40.545215 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d87f0ca-a0f5-406b-a975-8f57c94083cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:35:40.545513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d87f0ca-a0f5-406b-a975-8f57c94083cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:40.547142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d87f0ca-a0f5-406b-a975-8f57c94083cc admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:40.559871 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1086/2175] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:35:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:35:40.564532 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960196c1-6c20-41c2-b6c7-4c9a5a873dd9 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:40.574357 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960196c1-6c20-41c2-b6c7-4c9a5a873dd9 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:40.580053 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0460b8f9-e07f-4bcd-b4d4-756e79663ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:40.580918 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460b8f9-e07f-4bcd-b4d4-756e79663ce4 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:40.581362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460b8f9-e07f-4bcd-b4d4-756e79663ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:40.581790 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460b8f9-e07f-4bcd-b4d4-756e79663ce4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:40.582450 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0460b8f9-e07f-4bcd-b4d4-756e79663ce4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:40.584533 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0460b8f9-e07f-4bcd-b4d4-756e79663ce4 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:40.616657 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1090/2176] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:40 2022] POST /identity/v3/projects => generated 367 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:40.621320 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d87f0ca-a0f5-406b-a975-8f57c94083cc admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:40.637325 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d87f0ca-a0f5-406b-a975-8f57c94083cc admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:40.730722 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b03536-8b5a-406d-a649-bc13df6ddf01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:40.731632 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b03536-8b5a-406d-a649-bc13df6ddf01 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:40.738946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b03536-8b5a-406d-a649-bc13df6ddf01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:40.739449 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b03536-8b5a-406d-a649-bc13df6ddf01 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:40.740416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b03536-8b5a-406d-a649-bc13df6ddf01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:40.743387 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b03536-8b5a-406d-a649-bc13df6ddf01 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:40.762649 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:40.771859 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1087/2177] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:40 2022] POST /identity/v3/users => generated 284 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:40.781603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0460b8f9-e07f-4bcd-b4d4-756e79663ce4 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:40.793124 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0460b8f9-e07f-4bcd-b4d4-756e79663ce4 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:40.797660 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a465dd-031a-4b1a-9471-67b6fbeb6bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:40.798292 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a465dd-031a-4b1a-9471-67b6fbeb6bbf admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:40.798725 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a465dd-031a-4b1a-9471-67b6fbeb6bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:40.799084 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a465dd-031a-4b1a-9471-67b6fbeb6bbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:40.799667 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a465dd-031a-4b1a-9471-67b6fbeb6bbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:40.801240 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a465dd-031a-4b1a-9471-67b6fbeb6bbf admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:40.814314 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1091/2178] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:40.820837 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b03536-8b5a-406d-a649-bc13df6ddf01 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:40.834392 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b03536-8b5a-406d-a649-bc13df6ddf01 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:40.844936 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685c8ea-e225-42fa-86f4-8d16e454eddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:40.846411 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685c8ea-e225-42fa-86f4-8d16e454eddf admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:40.846804 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685c8ea-e225-42fa-86f4-8d16e454eddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:40.847174 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685c8ea-e225-42fa-86f4-8d16e454eddf admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:40.847956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685c8ea-e225-42fa-86f4-8d16e454eddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:40.850841 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685c8ea-e225-42fa-86f4-8d16e454eddf admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:40.868023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1088/2179] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:40 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:40.870939 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a465dd-031a-4b1a-9471-67b6fbeb6bbf admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:40.882004 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a465dd-031a-4b1a-9471-67b6fbeb6bbf admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:40.887884 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f36aa-c631-4838-97af-871b984b154e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:40.888641 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f36aa-c631-4838-97af-871b984b154e admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:40.888932 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f36aa-c631-4838-97af-871b984b154e admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:40.889236 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f36aa-c631-4838-97af-871b984b154e admin admin] PATH_INFO: `/v3/projects/a800ab7dba884926963b6f95cd1dc3b5/users/06b15e6b529c4fee8e0fbd8ff85181da/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:40.911431 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f36aa-c631-4838-97af-871b984b154e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a800ab7dba884926963b6f95cd1dc3b5, user_id=06b15e6b529c4fee8e0fbd8ff85181da, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:40.914073 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f36aa-c631-4838-97af-871b984b154e admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:40.931894 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1092/2180] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:40 2022] PUT /identity/v3/projects/a800ab7dba884926963b6f95cd1dc3b5/users/06b15e6b529c4fee8e0fbd8ff85181da/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:40.942176 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e85fd7-bcfb-4cfa-80de-97b08a5cacd0 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:40.942510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e85fd7-bcfb-4cfa-80de-97b08a5cacd0 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:40.943390 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e85fd7-bcfb-4cfa-80de-97b08a5cacd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:40.980387 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-91e85fd7-bcfb-4cfa-80de-97b08a5cacd0 None None] MFA Rules not processed for user `06b15e6b529c4fee8e0fbd8ff85181da`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:40.983086 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-91e85fd7-bcfb-4cfa-80de-97b08a5cacd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:41.060335 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1089/2181] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:35:41.064646 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f36aa-c631-4838-97af-871b984b154e admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:41.128002 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f36aa-c631-4838-97af-871b984b154e admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:41.133275 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38572aa-232b-40fd-9c4d-a8b4214cac28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:41.133963 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38572aa-232b-40fd-9c4d-a8b4214cac28 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:41.134366 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38572aa-232b-40fd-9c4d-a8b4214cac28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:41.134643 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38572aa-232b-40fd-9c4d-a8b4214cac28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:41.135274 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38572aa-232b-40fd-9c4d-a8b4214cac28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:41.137161 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38572aa-232b-40fd-9c4d-a8b4214cac28 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:41.167865 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1093/2182] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:41 2022] POST /identity/v3/projects => generated 369 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:41.171718 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e85fd7-bcfb-4cfa-80de-97b08a5cacd0 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:41.187240 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e85fd7-bcfb-4cfa-80de-97b08a5cacd0 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:41.252629 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce626210-42fa-4d3e-b9b0-ffa0627c15ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:41.253706 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce626210-42fa-4d3e-b9b0-ffa0627c15ce admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:41.253706 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce626210-42fa-4d3e-b9b0-ffa0627c15ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:41.253956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce626210-42fa-4d3e-b9b0-ffa0627c15ce admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:41.254474 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce626210-42fa-4d3e-b9b0-ffa0627c15ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:41.256400 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce626210-42fa-4d3e-b9b0-ffa0627c15ce admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:41.277722 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1090/2183] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:41 2022] POST /identity/v3/users => generated 291 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:41.281721 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38572aa-232b-40fd-9c4d-a8b4214cac28 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:41.293219 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38572aa-232b-40fd-9c4d-a8b4214cac28 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:41.297856 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f294cb80-e910-4a42-bda0-8296013e45f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:41.298501 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294cb80-e910-4a42-bda0-8296013e45f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:41.298786 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294cb80-e910-4a42-bda0-8296013e45f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:41.299059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294cb80-e910-4a42-bda0-8296013e45f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:41.299731 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f294cb80-e910-4a42-bda0-8296013e45f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:41.301303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f294cb80-e910-4a42-bda0-8296013e45f7 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:41.312439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1094/2184] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:41.316576 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce626210-42fa-4d3e-b9b0-ffa0627c15ce admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:41.326833 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce626210-42fa-4d3e-b9b0-ffa0627c15ce admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:41.331549 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b7762-ee01-4222-966d-279e21ef1ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:41.332324 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b7762-ee01-4222-966d-279e21ef1ee0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:41.332664 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b7762-ee01-4222-966d-279e21ef1ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:41.332949 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b7762-ee01-4222-966d-279e21ef1ee0 admin admin] PATH_INFO: `/v3/projects/9f0caf8ee98046b0b35d573a3843a22f/users/d38057b3fa884716bd392ecb7c5702e6/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:41.348558 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256b7762-ee01-4222-966d-279e21ef1ee0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f0caf8ee98046b0b35d573a3843a22f, user_id=d38057b3fa884716bd392ecb7c5702e6, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:41.351362 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256b7762-ee01-4222-966d-279e21ef1ee0 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:41.365209 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1091/2185] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:41 2022] PUT /identity/v3/projects/9f0caf8ee98046b0b35d573a3843a22f/users/d38057b3fa884716bd392ecb7c5702e6/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:41.370079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f294cb80-e910-4a42-bda0-8296013e45f7 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:41.380466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f294cb80-e910-4a42-bda0-8296013e45f7 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:41.446472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264b09b-460a-4474-9e7f-917156186ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:41.447190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264b09b-460a-4474-9e7f-917156186ae0 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:41.447519 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264b09b-460a-4474-9e7f-917156186ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:41.447860 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264b09b-460a-4474-9e7f-917156186ae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:41.448578 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c264b09b-460a-4474-9e7f-917156186ae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:41.450222 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c264b09b-460a-4474-9e7f-917156186ae0 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:41.452085 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:41.462269 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1095/2186] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:35:41.465646 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b7762-ee01-4222-966d-279e21ef1ee0 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:41.485034 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b7762-ee01-4222-966d-279e21ef1ee0 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:41.492084 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df35565-009a-4538-81df-2dc7279d196f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:41.493562 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df35565-009a-4538-81df-2dc7279d196f admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:41.500545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df35565-009a-4538-81df-2dc7279d196f admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:41.512426 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df35565-009a-4538-81df-2dc7279d196f admin admin] PATH_INFO: `/v3/projects/9f0caf8ee98046b0b35d573a3843a22f/users/d38057b3fa884716bd392ecb7c5702e6/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:41.518890 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df35565-009a-4538-81df-2dc7279d196f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f0caf8ee98046b0b35d573a3843a22f, user_id=d38057b3fa884716bd392ecb7c5702e6, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:41.523260 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df35565-009a-4538-81df-2dc7279d196f admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:41.536103 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1092/2187] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:41 2022] PUT /identity/v3/projects/9f0caf8ee98046b0b35d573a3843a22f/users/d38057b3fa884716bd392ecb7c5702e6/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:41.549069 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7460f7d-c28f-4e70-8769-55a6893aae55 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:41.549443 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7460f7d-c28f-4e70-8769-55a6893aae55 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:41.549810 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7460f7d-c28f-4e70-8769-55a6893aae55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:41.585613 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-e7460f7d-c28f-4e70-8769-55a6893aae55 None None] MFA Rules not processed for user `d38057b3fa884716bd392ecb7c5702e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:41.586892 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-e7460f7d-c28f-4e70-8769-55a6893aae55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:41.659416 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1096/2188] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:41 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:41.672320 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df35565-009a-4538-81df-2dc7279d196f admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:41.674373 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-2df35565-009a-4538-81df-2dc7279d196f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:41.708674 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:41.738394 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df35565-009a-4538-81df-2dc7279d196f admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:41.742306 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebbd41c-c836-4eb9-8332-bf0b6eae1568 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:41.743509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbd41c-c836-4eb9-8332-bf0b6eae1568 service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:41.743748 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbd41c-c836-4eb9-8332-bf0b6eae1568 service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:41.744019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbd41c-c836-4eb9-8332-bf0b6eae1568 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:41.755803 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebbd41c-c836-4eb9-8332-bf0b6eae1568 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:41.757342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebbd41c-c836-4eb9-8332-bf0b6eae1568 service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:41.768828 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1093/2189] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:41 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:43.143380 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7460f7d-c28f-4e70-8769-55a6893aae55 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:43.203874 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7460f7d-c28f-4e70-8769-55a6893aae55 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:43.208218 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11880c9-12f5-4782-a299-1368485aea5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:43.208842 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11880c9-12f5-4782-a299-1368485aea5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:43.209121 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11880c9-12f5-4782-a299-1368485aea5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:43.209393 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11880c9-12f5-4782-a299-1368485aea5c admin admin] PATH_INFO: `/v3/users/06b15e6b529c4fee8e0fbd8ff85181da` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:43.210524 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11880c9-12f5-4782-a299-1368485aea5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=06b15e6b529c4fee8e0fbd8ff85181da)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:43.212421 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11880c9-12f5-4782-a299-1368485aea5c admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:43.266836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f11880c9-12f5-4782-a299-1368485aea5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06b15e6b529c4fee8e0fbd8ff85181da'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:43.275224 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f11880c9-12f5-4782-a299-1368485aea5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06b15e6b529c4fee8e0fbd8ff85181da'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:43.276036 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f11880c9-12f5-4782-a299-1368485aea5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06b15e6b529c4fee8e0fbd8ff85181da'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:43.288520 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f11880c9-12f5-4782-a299-1368485aea5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06b15e6b529c4fee8e0fbd8ff85181da'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:43.289154 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f11880c9-12f5-4782-a299-1368485aea5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06b15e6b529c4fee8e0fbd8ff85181da'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:43.297814 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:43.302619 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1097/2190] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:43 2022] DELETE /identity/v3/users/06b15e6b529c4fee8e0fbd8ff85181da => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:43.307803 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24daa69e-891c-4e76-bb21-23624a3db21d None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:43.308321 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24daa69e-891c-4e76-bb21-23624a3db21d None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:43.308413 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24daa69e-891c-4e76-bb21-23624a3db21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:43.326862 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-24daa69e-891c-4e76-bb21-23624a3db21d None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:43.328047 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-24daa69e-891c-4e76-bb21-23624a3db21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:43.387204 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1094/2191] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:35:43.396766 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11880c9-12f5-4782-a299-1368485aea5c admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:43.398169 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f11880c9-12f5-4782-a299-1368485aea5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:43.456877 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11880c9-12f5-4782-a299-1368485aea5c admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:43.461163 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681927ee-8c07-4228-9ebe-4e11d8225356 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:43.461855 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681927ee-8c07-4228-9ebe-4e11d8225356 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:43.462187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681927ee-8c07-4228-9ebe-4e11d8225356 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:43.462500 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681927ee-8c07-4228-9ebe-4e11d8225356 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:43.472727 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681927ee-8c07-4228-9ebe-4e11d8225356 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:43.474161 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681927ee-8c07-4228-9ebe-4e11d8225356 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:43.484341 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1098/2192] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:43.754086 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24daa69e-891c-4e76-bb21-23624a3db21d None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:43.755522 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-24daa69e-891c-4e76-bb21-23624a3db21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:43.769872 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24daa69e-891c-4e76-bb21-23624a3db21d None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:43.775583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f7935a-1f54-42c5-87d5-1a2218934281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:43.776358 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7935a-1f54-42c5-87d5-1a2218934281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:43.776719 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7935a-1f54-42c5-87d5-1a2218934281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:43.777064 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7935a-1f54-42c5-87d5-1a2218934281 admin admin] PATH_INFO: `/v3/projects/a800ab7dba884926963b6f95cd1dc3b5` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:43.778294 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f7935a-1f54-42c5-87d5-1a2218934281 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a800ab7dba884926963b6f95cd1dc3b5)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:43.780497 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f7935a-1f54-42c5-87d5-1a2218934281 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:43.835566 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-86f7935a-1f54-42c5-87d5-1a2218934281 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a800ab7dba884926963b6f95cd1dc3b5', 'request_id': 'req-86f7935a-1f54-42c5-87d5-1a2218934281'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:43.845400 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-86f7935a-1f54-42c5-87d5-1a2218934281 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a800ab7dba884926963b6f95cd1dc3b5', 'request_id': 'req-86f7935a-1f54-42c5-87d5-1a2218934281'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:43.853135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-86f7935a-1f54-42c5-87d5-1a2218934281 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a800ab7dba884926963b6f95cd1dc3b5', 'request_id': 'req-86f7935a-1f54-42c5-87d5-1a2218934281'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:43.864870 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-86f7935a-1f54-42c5-87d5-1a2218934281 admin admin] The token cache is being invalidate because project a800ab7dba884926963b6f95cd1dc3b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:43.865355 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-86f7935a-1f54-42c5-87d5-1a2218934281 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:43.866181 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-86f7935a-1f54-42c5-87d5-1a2218934281 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:43.871990 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1095/2193] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:43 2022] DELETE /identity/v3/projects/a800ab7dba884926963b6f95cd1dc3b5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:43.873274 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681927ee-8c07-4228-9ebe-4e11d8225356 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:43.883912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681927ee-8c07-4228-9ebe-4e11d8225356 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:43.965381 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9a38d-e411-4b0e-8801-32809604e1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:43.965938 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9a38d-e411-4b0e-8801-32809604e1ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:43.966222 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9a38d-e411-4b0e-8801-32809604e1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:43.966553 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9a38d-e411-4b0e-8801-32809604e1ce admin admin] PATH_INFO: `/v3/users/d38057b3fa884716bd392ecb7c5702e6` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:43.967687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c9a38d-e411-4b0e-8801-32809604e1ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=d38057b3fa884716bd392ecb7c5702e6)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:43.969655 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c9a38d-e411-4b0e-8801-32809604e1ce admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:44.017743 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-48c9a38d-e411-4b0e-8801-32809604e1ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd38057b3fa884716bd392ecb7c5702e6'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:44.026094 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-48c9a38d-e411-4b0e-8801-32809604e1ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd38057b3fa884716bd392ecb7c5702e6'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:44.026911 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-48c9a38d-e411-4b0e-8801-32809604e1ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd38057b3fa884716bd392ecb7c5702e6'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:44.040525 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-48c9a38d-e411-4b0e-8801-32809604e1ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd38057b3fa884716bd392ecb7c5702e6'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:44.041223 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-48c9a38d-e411-4b0e-8801-32809604e1ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd38057b3fa884716bd392ecb7c5702e6'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:44.053554 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1099/2194] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:43 2022] DELETE /identity/v3/users/d38057b3fa884716bd392ecb7c5702e6 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:44.345947 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f7935a-1f54-42c5-87d5-1a2218934281 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:44.347733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-86f7935a-1f54-42c5-87d5-1a2218934281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:44.365293 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f7935a-1f54-42c5-87d5-1a2218934281 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:44.423461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:44.431704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee72e5-89e5-47fe-9e59-413b41906b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:44.432500 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee72e5-89e5-47fe-9e59-413b41906b4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:44.432808 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee72e5-89e5-47fe-9e59-413b41906b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:44.433116 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee72e5-89e5-47fe-9e59-413b41906b4f admin admin] PATH_INFO: `/v3/projects/9f0caf8ee98046b0b35d573a3843a22f` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:44.434625 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee72e5-89e5-47fe-9e59-413b41906b4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f0caf8ee98046b0b35d573a3843a22f)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:44.437071 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee72e5-89e5-47fe-9e59-413b41906b4f admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:44.486692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-17ee72e5-89e5-47fe-9e59-413b41906b4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f0caf8ee98046b0b35d573a3843a22f', 'request_id': 'req-17ee72e5-89e5-47fe-9e59-413b41906b4f'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:44.494763 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-17ee72e5-89e5-47fe-9e59-413b41906b4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f0caf8ee98046b0b35d573a3843a22f', 'request_id': 'req-17ee72e5-89e5-47fe-9e59-413b41906b4f'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:44.501019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-17ee72e5-89e5-47fe-9e59-413b41906b4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f0caf8ee98046b0b35d573a3843a22f', 'request_id': 'req-17ee72e5-89e5-47fe-9e59-413b41906b4f'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:44.512844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-17ee72e5-89e5-47fe-9e59-413b41906b4f admin admin] The token cache is being invalidate because project 9f0caf8ee98046b0b35d573a3843a22f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:44.513145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-17ee72e5-89e5-47fe-9e59-413b41906b4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:44.513816 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-17ee72e5-89e5-47fe-9e59-413b41906b4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:44.515401 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1096/2195] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:44 2022] DELETE /identity/v3/projects/9f0caf8ee98046b0b35d573a3843a22f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:44.520309 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa6bc0-0dd6-4335-8f00-6790d76cfce0 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:44.520800 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa6bc0-0dd6-4335-8f00-6790d76cfce0 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:44.520881 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa6bc0-0dd6-4335-8f00-6790d76cfce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:44.543913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-35fa6bc0-0dd6-4335-8f00-6790d76cfce0 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:44.545137 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-35fa6bc0-0dd6-4335-8f00-6790d76cfce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:44.561880 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:44.610044 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1100/2196] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:44.617626 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b5797-61ae-4dcd-992b-1d0003de95f8 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:44.617918 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b5797-61ae-4dcd-992b-1d0003de95f8 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:44.618197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b5797-61ae-4dcd-992b-1d0003de95f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:44.637979 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-285b5797-61ae-4dcd-992b-1d0003de95f8 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:44.639215 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-285b5797-61ae-4dcd-992b-1d0003de95f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:44.647224 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1097/2197] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:44.652060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa6bc0-0dd6-4335-8f00-6790d76cfce0 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:44.661831 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa6bc0-0dd6-4335-8f00-6790d76cfce0 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:44.667314 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20667e39-f0cf-490b-bfa0-e2a0cfd825c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:44.667910 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20667e39-f0cf-490b-bfa0-e2a0cfd825c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:44.668192 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20667e39-f0cf-490b-bfa0-e2a0cfd825c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:44.668492 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20667e39-f0cf-490b-bfa0-e2a0cfd825c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:44.669131 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20667e39-f0cf-490b-bfa0-e2a0cfd825c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:35:44.669433 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20667e39-f0cf-490b-bfa0-e2a0cfd825c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:44.671051 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20667e39-f0cf-490b-bfa0-e2a0cfd825c3 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:44.682610 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1101/2198] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:35:44.687065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b5797-61ae-4dcd-992b-1d0003de95f8 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:44.698131 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b5797-61ae-4dcd-992b-1d0003de95f8 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:44.702754 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68047cc-41c5-49a4-99d3-1a9450ae0ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:44.703342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68047cc-41c5-49a4-99d3-1a9450ae0ac5 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:44.703623 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68047cc-41c5-49a4-99d3-1a9450ae0ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:44.703896 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68047cc-41c5-49a4-99d3-1a9450ae0ac5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:44.704530 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68047cc-41c5-49a4-99d3-1a9450ae0ac5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:44.706470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68047cc-41c5-49a4-99d3-1a9450ae0ac5 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:44.733869 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1098/2199] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:44 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:44.738067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20667e39-f0cf-490b-bfa0-e2a0cfd825c3 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:44.747980 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20667e39-f0cf-490b-bfa0-e2a0cfd825c3 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:44.778464 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68047cc-41c5-49a4-99d3-1a9450ae0ac5 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:44.780604 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-b68047cc-41c5-49a4-99d3-1a9450ae0ac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:44.804657 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ba503-8f0b-4c89-8de1-cb9bd5f8e77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:44.805291 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ba503-8f0b-4c89-8de1-cb9bd5f8e77c admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:44.805577 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ba503-8f0b-4c89-8de1-cb9bd5f8e77c admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:44.805851 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ba503-8f0b-4c89-8de1-cb9bd5f8e77c admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:44.806502 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ba503-8f0b-4c89-8de1-cb9bd5f8e77c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:44.808412 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ba503-8f0b-4c89-8de1-cb9bd5f8e77c admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:44.830766 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1102/2200] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:44 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:44.834473 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ba503-8f0b-4c89-8de1-cb9bd5f8e77c admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:44.844810 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ba503-8f0b-4c89-8de1-cb9bd5f8e77c admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:44.845177 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68047cc-41c5-49a4-99d3-1a9450ae0ac5 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:44.849820 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640fc73-861d-4fac-9dd1-2a4ebbf6b161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:44.849959 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd072d-3508-454a-8ac1-347d470e294e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:44.850469 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640fc73-861d-4fac-9dd1-2a4ebbf6b161 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:44.850587 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd072d-3508-454a-8ac1-347d470e294e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:44.850772 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640fc73-861d-4fac-9dd1-2a4ebbf6b161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:44.850891 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd072d-3508-454a-8ac1-347d470e294e admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:44.851045 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd072d-3508-454a-8ac1-347d470e294e admin admin] PATH_INFO: `/v3/users/33b31c49ceaf4108b41d6b5ad12cd98c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:44.851185 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640fc73-861d-4fac-9dd1-2a4ebbf6b161 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:44.851678 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640fc73-861d-4fac-9dd1-2a4ebbf6b161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:44.852252 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd072d-3508-454a-8ac1-347d470e294e admin admin] RBAC: Authorizing `identity:delete_user(user_id=33b31c49ceaf4108b41d6b5ad12cd98c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:44.853280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640fc73-861d-4fac-9dd1-2a4ebbf6b161 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:44.854442 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd072d-3508-454a-8ac1-347d470e294e admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:44.864284 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1103/2201] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:44.868189 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640fc73-861d-4fac-9dd1-2a4ebbf6b161 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:44.877692 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640fc73-861d-4fac-9dd1-2a4ebbf6b161 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:44.882372 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6679e4-eb1f-4d0a-8ece-c52d75655fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:44.883474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6679e4-eb1f-4d0a-8ece-c52d75655fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:44.883768 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6679e4-eb1f-4d0a-8ece-c52d75655fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:44.884045 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6679e4-eb1f-4d0a-8ece-c52d75655fd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:44.884708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6679e4-eb1f-4d0a-8ece-c52d75655fd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:44.886352 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6679e4-eb1f-4d0a-8ece-c52d75655fd6 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:44.897034 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1104/2202] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:35:44.901766 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6679e4-eb1f-4d0a-8ece-c52d75655fd6 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:44.909437 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-d0bd072d-3508-454a-8ac1-347d470e294e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33b31c49ceaf4108b41d6b5ad12cd98c'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:44.910100 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-d0bd072d-3508-454a-8ac1-347d470e294e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33b31c49ceaf4108b41d6b5ad12cd98c'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:44.911433 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6679e4-eb1f-4d0a-8ece-c52d75655fd6 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:44.916979 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa5969-487c-4cd8-8474-e04306d7649d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:44.917636 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa5969-487c-4cd8-8474-e04306d7649d admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:44.917936 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa5969-487c-4cd8-8474-e04306d7649d admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:44.918359 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa5969-487c-4cd8-8474-e04306d7649d admin admin] PATH_INFO: `/v3/projects/ad55304fc333422b805a2c6564e0d8aa/users/2d57bda87fd84c3e9a444d020dd43b0b/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:44.918588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-d0bd072d-3508-454a-8ac1-347d470e294e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33b31c49ceaf4108b41d6b5ad12cd98c'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:44.919418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-d0bd072d-3508-454a-8ac1-347d470e294e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33b31c49ceaf4108b41d6b5ad12cd98c'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:44.929499 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:44.935872 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aa5969-487c-4cd8-8474-e04306d7649d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad55304fc333422b805a2c6564e0d8aa, user_id=2d57bda87fd84c3e9a444d020dd43b0b, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:44.939872 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aa5969-487c-4cd8-8474-e04306d7649d admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:44.940170 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-d0bd072d-3508-454a-8ac1-347d470e294e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33b31c49ceaf4108b41d6b5ad12cd98c'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:44.958349 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1105/2204] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:44 2022] PUT /identity/v3/projects/ad55304fc333422b805a2c6564e0d8aa/users/2d57bda87fd84c3e9a444d020dd43b0b/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:44.962387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343355c7-5a05-492c-ac4a-e21fe615bef2 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:44.962873 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343355c7-5a05-492c-ac4a-e21fe615bef2 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:44.963352 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343355c7-5a05-492c-ac4a-e21fe615bef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:44.970830 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1099/2204] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:44 2022] DELETE /identity/v3/users/33b31c49ceaf4108b41d6b5ad12cd98c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:44.972337 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba6aab-41d4-400e-b6cb-8261deb6a8a9 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:44.972679 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba6aab-41d4-400e-b6cb-8261deb6a8a9 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:44.972960 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba6aab-41d4-400e-b6cb-8261deb6a8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:44.984919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-343355c7-5a05-492c-ac4a-e21fe615bef2 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:44.986171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-343355c7-5a05-492c-ac4a-e21fe615bef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:45.002447 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-8eba6aab-41d4-400e-b6cb-8261deb6a8a9 None None] MFA Rules not processed for user `2d57bda87fd84c3e9a444d020dd43b0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:45.003736 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-8eba6aab-41d4-400e-b6cb-8261deb6a8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:45.053226 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:45.075829 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1106/2205] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:45.099937 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343355c7-5a05-492c-ac4a-e21fe615bef2 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:45.099937 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-343355c7-5a05-492c-ac4a-e21fe615bef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:45.112493 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1100/2206] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:45.112493 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba6aab-41d4-400e-b6cb-8261deb6a8a9 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:45.112493 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-8eba6aab-41d4-400e-b6cb-8261deb6a8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:45.165160 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343355c7-5a05-492c-ac4a-e21fe615bef2 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:45.168912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02250a8a-e745-4cfc-ad39-c6285d3f7f9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:45.169720 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02250a8a-e745-4cfc-ad39-c6285d3f7f9c service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:45.170063 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02250a8a-e745-4cfc-ad39-c6285d3f7f9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:45.170428 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02250a8a-e745-4cfc-ad39-c6285d3f7f9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:45.181497 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02250a8a-e745-4cfc-ad39-c6285d3f7f9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:45.184345 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02250a8a-e745-4cfc-ad39-c6285d3f7f9c service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:45.194252 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba6aab-41d4-400e-b6cb-8261deb6a8a9 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:45.198121 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1107/2207] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:35:45.198993 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664784f-8396-418d-a8ae-4f8393d6d23a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:45.199127 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02250a8a-e745-4cfc-ad39-c6285d3f7f9c service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:45.200152 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664784f-8396-418d-a8ae-4f8393d6d23a service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:45.200582 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664784f-8396-418d-a8ae-4f8393d6d23a service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:45.201083 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664784f-8396-418d-a8ae-4f8393d6d23a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:45.201898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-02250a8a-e745-4cfc-ad39-c6285d3f7f9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:45.203306 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-f664784f-8396-418d-a8ae-4f8393d6d23a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:45.215580 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02250a8a-e745-4cfc-ad39-c6285d3f7f9c service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:45.221249 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d16a6da-e221-414e-b60e-56da78ffc36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:45.222209 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16a6da-e221-414e-b60e-56da78ffc36f admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:45.222517 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16a6da-e221-414e-b60e-56da78ffc36f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:45.222808 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16a6da-e221-414e-b60e-56da78ffc36f admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:45.223555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d16a6da-e221-414e-b60e-56da78ffc36f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:45.225583 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d16a6da-e221-414e-b60e-56da78ffc36f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:45.273157 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1108/2208] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:45 2022] POST /identity/v3/projects => generated 377 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:45.277734 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d16a6da-e221-414e-b60e-56da78ffc36f admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:45.305555 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f664784f-8396-418d-a8ae-4f8393d6d23a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:45.307460 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f664784f-8396-418d-a8ae-4f8393d6d23a service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:45.329230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1101/2209] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:45 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:45.388280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d16a6da-e221-414e-b60e-56da78ffc36f admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:45.394428 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668a9b2-2a9a-4258-8329-daec5398a0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:45.395056 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668a9b2-2a9a-4258-8329-daec5398a0a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:45.395339 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668a9b2-2a9a-4258-8329-daec5398a0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:45.395804 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668a9b2-2a9a-4258-8329-daec5398a0a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:45.396451 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1668a9b2-2a9a-4258-8329-daec5398a0a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:45.398414 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1668a9b2-2a9a-4258-8329-daec5398a0a6 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:45.433896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1109/2210] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:45 2022] POST /identity/v3/users => generated 295 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:35:45.438775 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664784f-8396-418d-a8ae-4f8393d6d23a service cinder] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:45.458109 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664784f-8396-418d-a8ae-4f8393d6d23a service cinder] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:45.467219 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabd12a-0571-46df-bc0c-cf32487c1d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:45.468759 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabd12a-0571-46df-bc0c-cf32487c1d2c admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:45.469225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabd12a-0571-46df-bc0c-cf32487c1d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:45.469686 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabd12a-0571-46df-bc0c-cf32487c1d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:45.470779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acabd12a-0571-46df-bc0c-cf32487c1d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:45.473351 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acabd12a-0571-46df-bc0c-cf32487c1d2c admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:45.495073 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1102/2211] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:45.497001 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668a9b2-2a9a-4258-8329-daec5398a0a6 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:45.513681 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668a9b2-2a9a-4258-8329-daec5398a0a6 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:45.522828 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f79526-7a58-49e6-92fd-c906913d9b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:45.524140 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f79526-7a58-49e6-92fd-c906913d9b07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:45.524431 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f79526-7a58-49e6-92fd-c906913d9b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:45.524869 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f79526-7a58-49e6-92fd-c906913d9b07 admin admin] PATH_INFO: `/v3/projects/dffbb6be6a84460383ddc4abe5443975/users/a7fb5d5bb7ac4fea95ba957b73a23956/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:45.566656 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f79526-7a58-49e6-92fd-c906913d9b07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dffbb6be6a84460383ddc4abe5443975, user_id=a7fb5d5bb7ac4fea95ba957b73a23956, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:45.575365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f79526-7a58-49e6-92fd-c906913d9b07 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:45.597410 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabd12a-0571-46df-bc0c-cf32487c1d2c admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:45.600350 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-acabd12a-0571-46df-bc0c-cf32487c1d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:45.604341 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1110/2212] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:45 2022] PUT /identity/v3/projects/dffbb6be6a84460383ddc4abe5443975/users/a7fb5d5bb7ac4fea95ba957b73a23956/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:45.611028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f79526-7a58-49e6-92fd-c906913d9b07 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:45.630036 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:45.655374 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabd12a-0571-46df-bc0c-cf32487c1d2c admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:45.711701 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:45.755850 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f79526-7a58-49e6-92fd-c906913d9b07 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:45.765462 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655a2ea-5d73-4179-aed9-44c24ecbe426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:45.766899 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655a2ea-5d73-4179-aed9-44c24ecbe426 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:45.767290 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655a2ea-5d73-4179-aed9-44c24ecbe426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:45.767691 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655a2ea-5d73-4179-aed9-44c24ecbe426 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:45.768824 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3655a2ea-5d73-4179-aed9-44c24ecbe426 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:45.785188 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834a057-45f9-42fa-a33a-0bf3feeaf707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:45.785895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834a057-45f9-42fa-a33a-0bf3feeaf707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:45.786183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834a057-45f9-42fa-a33a-0bf3feeaf707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:45.786491 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834a057-45f9-42fa-a33a-0bf3feeaf707 admin admin] PATH_INFO: `/v3/projects/e0142c39dbb14868b0826834b575d539` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:45.788450 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834a057-45f9-42fa-a33a-0bf3feeaf707 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0142c39dbb14868b0826834b575d539)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:45.791403 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3655a2ea-5d73-4179-aed9-44c24ecbe426 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:45.791559 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834a057-45f9-42fa-a33a-0bf3feeaf707 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:45.810489 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1111/2213] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:45.812539 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655a2ea-5d73-4179-aed9-44c24ecbe426 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:45.824571 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655a2ea-5d73-4179-aed9-44c24ecbe426 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:45.830783 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce6aec-9118-4bdc-87c0-72b89b29b560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:45.831499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce6aec-9118-4bdc-87c0-72b89b29b560 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:45.831752 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce6aec-9118-4bdc-87c0-72b89b29b560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:45.832032 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce6aec-9118-4bdc-87c0-72b89b29b560 admin admin] PATH_INFO: `/v3/projects/dffbb6be6a84460383ddc4abe5443975/users/a7fb5d5bb7ac4fea95ba957b73a23956/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:45.835034 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce6aec-9118-4bdc-87c0-72b89b29b560 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dffbb6be6a84460383ddc4abe5443975, user_id=a7fb5d5bb7ac4fea95ba957b73a23956, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:45.839891 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce6aec-9118-4bdc-87c0-72b89b29b560 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:45.854409 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1112/2214] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:45 2022] PUT /identity/v3/projects/dffbb6be6a84460383ddc4abe5443975/users/a7fb5d5bb7ac4fea95ba957b73a23956/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:45.865177 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab031ad-6454-44c4-860d-0d4f4be8302e None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:45.867085 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab031ad-6454-44c4-860d-0d4f4be8302e None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:45.867741 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab031ad-6454-44c4-860d-0d4f4be8302e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:45.872053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c834a057-45f9-42fa-a33a-0bf3feeaf707 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0142c39dbb14868b0826834b575d539', 'request_id': 'req-c834a057-45f9-42fa-a33a-0bf3feeaf707'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:45.881796 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c834a057-45f9-42fa-a33a-0bf3feeaf707 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0142c39dbb14868b0826834b575d539', 'request_id': 'req-c834a057-45f9-42fa-a33a-0bf3feeaf707'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:45.891375 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c834a057-45f9-42fa-a33a-0bf3feeaf707 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0142c39dbb14868b0826834b575d539', 'request_id': 'req-c834a057-45f9-42fa-a33a-0bf3feeaf707'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:45.906015 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c834a057-45f9-42fa-a33a-0bf3feeaf707 admin admin] The token cache is being invalidate because project e0142c39dbb14868b0826834b575d539 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:45.906298 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c834a057-45f9-42fa-a33a-0bf3feeaf707 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:45.906967 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c834a057-45f9-42fa-a33a-0bf3feeaf707 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:45.908499 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1103/2215] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:45 2022] DELETE /identity/v3/projects/e0142c39dbb14868b0826834b575d539 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:45.915005 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec29d80-4bfe-4f49-9f72-ab3e33ab9c97 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:45.915398 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec29d80-4bfe-4f49-9f72-ab3e33ab9c97 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:45.915770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec29d80-4bfe-4f49-9f72-ab3e33ab9c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:45.920961 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-bab031ad-6454-44c4-860d-0d4f4be8302e None None] MFA Rules not processed for user `a7fb5d5bb7ac4fea95ba957b73a23956`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:45.922368 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-bab031ad-6454-44c4-860d-0d4f4be8302e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:45.937928 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-4ec29d80-4bfe-4f49-9f72-ab3e33ab9c97 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:45.939702 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-4ec29d80-4bfe-4f49-9f72-ab3e33ab9c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:46.000394 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1113/2216] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:45 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:46.011364 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab031ad-6454-44c4-860d-0d4f4be8302e None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:46.012695 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-bab031ad-6454-44c4-860d-0d4f4be8302e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:46.014044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1104/2217] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:46.021614 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d93d47-23a1-4307-939c-5dd0b47b7089 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:46.021901 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d93d47-23a1-4307-939c-5dd0b47b7089 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:46.022177 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d93d47-23a1-4307-939c-5dd0b47b7089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:46.044267 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-38d93d47-23a1-4307-939c-5dd0b47b7089 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:46.045460 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-38d93d47-23a1-4307-939c-5dd0b47b7089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:46.053327 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1105/2218] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:35:46.057545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d93d47-23a1-4307-939c-5dd0b47b7089 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:46.066706 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d93d47-23a1-4307-939c-5dd0b47b7089 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:46.072602 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30dac9b-6a30-422e-aa18-436eb3828c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:46.073196 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30dac9b-6a30-422e-aa18-436eb3828c80 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:46.073475 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30dac9b-6a30-422e-aa18-436eb3828c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:46.074603 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30dac9b-6a30-422e-aa18-436eb3828c80 admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:46.074603 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30dac9b-6a30-422e-aa18-436eb3828c80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:35:46.074790 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30dac9b-6a30-422e-aa18-436eb3828c80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:46.075175 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab031ad-6454-44c4-860d-0d4f4be8302e None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:46.076351 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30dac9b-6a30-422e-aa18-436eb3828c80 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:46.080675 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8393a-7be8-43d0-9a0b-99ddccd092db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:46.081605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8393a-7be8-43d0-9a0b-99ddccd092db service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:46.082004 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8393a-7be8-43d0-9a0b-99ddccd092db service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:46.082523 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8393a-7be8-43d0-9a0b-99ddccd092db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:46.089848 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1106/2219] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:35:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:35:46.095421 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30dac9b-6a30-422e-aa18-436eb3828c80 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:46.095798 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb8393a-7be8-43d0-9a0b-99ddccd092db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:46.097944 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb8393a-7be8-43d0-9a0b-99ddccd092db service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:46.112016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30dac9b-6a30-422e-aa18-436eb3828c80 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:46.114042 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1114/2220] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:46 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:35:46.117938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b8db0-2e26-4308-bf34-3ed6f2067109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:46.118619 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b8db0-2e26-4308-bf34-3ed6f2067109 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:46.118945 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b8db0-2e26-4308-bf34-3ed6f2067109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:46.119637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b8db0-2e26-4308-bf34-3ed6f2067109 admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:46.120079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b8db0-2e26-4308-bf34-3ed6f2067109 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:46.122252 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b8db0-2e26-4308-bf34-3ed6f2067109 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:46.139845 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:46.163157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1107/2221] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:46 2022] POST /identity/v3/projects => generated 367 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:35:46.167761 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8393a-7be8-43d0-9a0b-99ddccd092db service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:46.180972 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8393a-7be8-43d0-9a0b-99ddccd092db service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:46.247958 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:46.262987 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b71353-33ed-49d4-a99a-ff67cf671676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:46.263732 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b71353-33ed-49d4-a99a-ff67cf671676 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:46.264118 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b71353-33ed-49d4-a99a-ff67cf671676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:46.264399 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b71353-33ed-49d4-a99a-ff67cf671676 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:46.265044 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b71353-33ed-49d4-a99a-ff67cf671676 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:46.267095 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b71353-33ed-49d4-a99a-ff67cf671676 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:46.295708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1115/2222] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:46 2022] POST /identity/v3/users => generated 284 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:35:46.299612 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b8db0-2e26-4308-bf34-3ed6f2067109 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:46.313829 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b8db0-2e26-4308-bf34-3ed6f2067109 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:46.323227 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab9beb0-d553-4f8a-b38f-fe8895258bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:46.323861 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9beb0-d553-4f8a-b38f-fe8895258bbe admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:46.324345 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9beb0-d553-4f8a-b38f-fe8895258bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:46.324630 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9beb0-d553-4f8a-b38f-fe8895258bbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:46.325295 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab9beb0-d553-4f8a-b38f-fe8895258bbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:46.326977 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab9beb0-d553-4f8a-b38f-fe8895258bbe admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:46.339654 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1108/2223] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:35:46.347859 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b71353-33ed-49d4-a99a-ff67cf671676 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:46.360189 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b71353-33ed-49d4-a99a-ff67cf671676 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:46.371462 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5893b-7781-488c-9afb-fa161f008961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:46.372209 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5893b-7781-488c-9afb-fa161f008961 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:46.372593 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5893b-7781-488c-9afb-fa161f008961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:46.372961 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5893b-7781-488c-9afb-fa161f008961 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:46.373707 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5893b-7781-488c-9afb-fa161f008961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:46.375432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5893b-7781-488c-9afb-fa161f008961 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:46.387074 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1116/2224] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:35:46.391033 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab9beb0-d553-4f8a-b38f-fe8895258bbe admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:46.402165 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab9beb0-d553-4f8a-b38f-fe8895258bbe admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:46.409848 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb9c705-b562-4e35-bd9a-c768f0e6e601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:46.411106 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9c705-b562-4e35-bd9a-c768f0e6e601 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:46.411488 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9c705-b562-4e35-bd9a-c768f0e6e601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:46.411947 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9c705-b562-4e35-bd9a-c768f0e6e601 admin admin] PATH_INFO: `/v3/projects/9525e7b503134f3da12368d42261e9be/users/74ce31b69cd04646843587b7de26c25b/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:46.430126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb9c705-b562-4e35-bd9a-c768f0e6e601 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9525e7b503134f3da12368d42261e9be, user_id=74ce31b69cd04646843587b7de26c25b, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:46.432640 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb9c705-b562-4e35-bd9a-c768f0e6e601 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:46.448929 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1109/2225] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:46 2022] PUT /identity/v3/projects/9525e7b503134f3da12368d42261e9be/users/74ce31b69cd04646843587b7de26c25b/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:46.464548 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb90da-895a-49ba-b20f-7b0405296140 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:46.464992 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb90da-895a-49ba-b20f-7b0405296140 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:46.465362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb90da-895a-49ba-b20f-7b0405296140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:46.502462 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-1bcb90da-895a-49ba-b20f-7b0405296140 None None] MFA Rules not processed for user `74ce31b69cd04646843587b7de26c25b`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:46.504762 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-1bcb90da-895a-49ba-b20f-7b0405296140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:46.576847 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb9c705-b562-4e35-bd9a-c768f0e6e601 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:46.578947 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-4bb9c705-b562-4e35-bd9a-c768f0e6e601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:46.585360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1117/2226] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:46.604563 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb90da-895a-49ba-b20f-7b0405296140 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:46.606073 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-1bcb90da-895a-49ba-b20f-7b0405296140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:46.663871 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb9c705-b562-4e35-bd9a-c768f0e6e601 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:46.676025 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ce667-5bdb-4ebe-8608-2d62a6011957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:46.677470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ce667-5bdb-4ebe-8608-2d62a6011957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:46.678008 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ce667-5bdb-4ebe-8608-2d62a6011957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:46.678808 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ce667-5bdb-4ebe-8608-2d62a6011957 admin admin] PATH_INFO: `/v3/users/2d57bda87fd84c3e9a444d020dd43b0b` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:46.681803 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ce667-5bdb-4ebe-8608-2d62a6011957 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d57bda87fd84c3e9a444d020dd43b0b)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:46.684143 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ce667-5bdb-4ebe-8608-2d62a6011957 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:46.696678 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb90da-895a-49ba-b20f-7b0405296140 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:46.701273 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003c0fe-407f-4a58-8f4e-b0f84561d9a1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b09f6d550c9142bf956b225bf5172e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:46.702182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003c0fe-407f-4a58-8f4e-b0f84561d9a1 service glance] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:46.702691 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003c0fe-407f-4a58-8f4e-b0f84561d9a1 service glance] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:46.703036 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003c0fe-407f-4a58-8f4e-b0f84561d9a1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:46.720090 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003c0fe-407f-4a58-8f4e-b0f84561d9a1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:46.721685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003c0fe-407f-4a58-8f4e-b0f84561d9a1 service glance] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:46.741653 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1118/2227] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:35:46.763614 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-746ce667-5bdb-4ebe-8608-2d62a6011957 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d57bda87fd84c3e9a444d020dd43b0b'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:46.764896 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-746ce667-5bdb-4ebe-8608-2d62a6011957 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d57bda87fd84c3e9a444d020dd43b0b'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:46.776904 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-746ce667-5bdb-4ebe-8608-2d62a6011957 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d57bda87fd84c3e9a444d020dd43b0b'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:46.778595 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-746ce667-5bdb-4ebe-8608-2d62a6011957 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d57bda87fd84c3e9a444d020dd43b0b'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:46.798710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-746ce667-5bdb-4ebe-8608-2d62a6011957 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d57bda87fd84c3e9a444d020dd43b0b'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:46.819782 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1110/2228] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:46 2022] DELETE /identity/v3/users/2d57bda87fd84c3e9a444d020dd43b0b => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:46.823001 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ce2e1-b5c9-4e6c-a073-55e0d718db98 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:46.823682 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ce2e1-b5c9-4e6c-a073-55e0d718db98 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:46.824076 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ce2e1-b5c9-4e6c-a073-55e0d718db98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:46.858515 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-d15ce2e1-b5c9-4e6c-a073-55e0d718db98 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:46.859936 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d15ce2e1-b5c9-4e6c-a073-55e0d718db98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:46.909674 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:46.940575 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1119/2229] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:46.951121 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ce667-5bdb-4ebe-8608-2d62a6011957 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:46.952912 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-746ce667-5bdb-4ebe-8608-2d62a6011957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:47.042906 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:47.068299 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ce667-5bdb-4ebe-8608-2d62a6011957 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:47.073350 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d006b3-a176-4218-a624-1260e28a1f7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:47.074807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d006b3-a176-4218-a624-1260e28a1f7d service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:47.075292 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d006b3-a176-4218-a624-1260e28a1f7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:47.075935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d006b3-a176-4218-a624-1260e28a1f7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:47.114149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d006b3-a176-4218-a624-1260e28a1f7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:47.116485 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d006b3-a176-4218-a624-1260e28a1f7d service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:47.141705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1111/2230] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:47.604382 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ce2e1-b5c9-4e6c-a073-55e0d718db98 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:47.605917 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d15ce2e1-b5c9-4e6c-a073-55e0d718db98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:47.618829 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ce2e1-b5c9-4e6c-a073-55e0d718db98 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:47.624183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed70698-c96a-452f-aa61-597014f3f64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:47.624737 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed70698-c96a-452f-aa61-597014f3f64c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:47.625116 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed70698-c96a-452f-aa61-597014f3f64c admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:47.625605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed70698-c96a-452f-aa61-597014f3f64c admin admin] PATH_INFO: `/v3/projects/ad55304fc333422b805a2c6564e0d8aa` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:47.627336 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed70698-c96a-452f-aa61-597014f3f64c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad55304fc333422b805a2c6564e0d8aa)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:47.629459 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed70698-c96a-452f-aa61-597014f3f64c admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:47.682601 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-4ed70698-c96a-452f-aa61-597014f3f64c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad55304fc333422b805a2c6564e0d8aa', 'request_id': 'req-4ed70698-c96a-452f-aa61-597014f3f64c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:47.688831 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-4ed70698-c96a-452f-aa61-597014f3f64c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad55304fc333422b805a2c6564e0d8aa', 'request_id': 'req-4ed70698-c96a-452f-aa61-597014f3f64c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:47.696987 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-4ed70698-c96a-452f-aa61-597014f3f64c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad55304fc333422b805a2c6564e0d8aa', 'request_id': 'req-4ed70698-c96a-452f-aa61-597014f3f64c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:47.707563 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-4ed70698-c96a-452f-aa61-597014f3f64c admin admin] The token cache is being invalidate because project ad55304fc333422b805a2c6564e0d8aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:47.708043 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-4ed70698-c96a-452f-aa61-597014f3f64c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:47.708721 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-4ed70698-c96a-452f-aa61-597014f3f64c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:47.710477 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1120/2231] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:47 2022] DELETE /identity/v3/projects/ad55304fc333422b805a2c6564e0d8aa => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:47.714538 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d006b3-a176-4218-a624-1260e28a1f7d service neutron] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:47.724511 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d006b3-a176-4218-a624-1260e28a1f7d service neutron] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:47.780515 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b234c7-fcc6-4ace-90eb-62109391b22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:47.781154 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b234c7-fcc6-4ace-90eb-62109391b22e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:47.781437 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b234c7-fcc6-4ace-90eb-62109391b22e admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:47.781713 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b234c7-fcc6-4ace-90eb-62109391b22e admin admin] PATH_INFO: `/v3/users/a7fb5d5bb7ac4fea95ba957b73a23956` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:47.782878 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b234c7-fcc6-4ace-90eb-62109391b22e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7fb5d5bb7ac4fea95ba957b73a23956)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:47.784837 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b234c7-fcc6-4ace-90eb-62109391b22e admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:47.832728 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-82b234c7-fcc6-4ace-90eb-62109391b22e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7fb5d5bb7ac4fea95ba957b73a23956'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:47.833306 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-82b234c7-fcc6-4ace-90eb-62109391b22e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7fb5d5bb7ac4fea95ba957b73a23956'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:47.842685 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-82b234c7-fcc6-4ace-90eb-62109391b22e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7fb5d5bb7ac4fea95ba957b73a23956'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:47.843320 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-82b234c7-fcc6-4ace-90eb-62109391b22e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7fb5d5bb7ac4fea95ba957b73a23956'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:47.858188 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-82b234c7-fcc6-4ace-90eb-62109391b22e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7fb5d5bb7ac4fea95ba957b73a23956'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:47.870019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1112/2232] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:47 2022] DELETE /identity/v3/users/a7fb5d5bb7ac4fea95ba957b73a23956 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:48.145293 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed70698-c96a-452f-aa61-597014f3f64c admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:48.146434 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-4ed70698-c96a-452f-aa61-597014f3f64c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:48.159994 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed70698-c96a-452f-aa61-597014f3f64c admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:48.223963 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f820e884-6aa9-497c-b0a4-fcf32a8b30f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:48.224669 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820e884-6aa9-497c-b0a4-fcf32a8b30f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:48.225070 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820e884-6aa9-497c-b0a4-fcf32a8b30f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:48.225424 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820e884-6aa9-497c-b0a4-fcf32a8b30f1 admin admin] PATH_INFO: `/v3/projects/dffbb6be6a84460383ddc4abe5443975` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:48.227299 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f820e884-6aa9-497c-b0a4-fcf32a8b30f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dffbb6be6a84460383ddc4abe5443975)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:48.232234 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f820e884-6aa9-497c-b0a4-fcf32a8b30f1 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:48.291211 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f820e884-6aa9-497c-b0a4-fcf32a8b30f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dffbb6be6a84460383ddc4abe5443975', 'request_id': 'req-f820e884-6aa9-497c-b0a4-fcf32a8b30f1'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:48.300240 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f820e884-6aa9-497c-b0a4-fcf32a8b30f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dffbb6be6a84460383ddc4abe5443975', 'request_id': 'req-f820e884-6aa9-497c-b0a4-fcf32a8b30f1'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:48.311838 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f820e884-6aa9-497c-b0a4-fcf32a8b30f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dffbb6be6a84460383ddc4abe5443975', 'request_id': 'req-f820e884-6aa9-497c-b0a4-fcf32a8b30f1'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:48.318637 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:48.327319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f820e884-6aa9-497c-b0a4-fcf32a8b30f1 admin admin] The token cache is being invalidate because project dffbb6be6a84460383ddc4abe5443975 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:48.327803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f820e884-6aa9-497c-b0a4-fcf32a8b30f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:48.328540 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f820e884-6aa9-497c-b0a4-fcf32a8b30f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:48.330208 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1121/2233] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:48 2022] DELETE /identity/v3/projects/dffbb6be6a84460383ddc4abe5443975 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:48.335649 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8ec8e-f8da-4516-9711-d9e1759e9a9a None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:48.335933 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8ec8e-f8da-4516-9711-d9e1759e9a9a None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:48.337097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8ec8e-f8da-4516-9711-d9e1759e9a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:48.392344 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-04b8ec8e-f8da-4516-9711-d9e1759e9a9a None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:48.393622 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-04b8ec8e-f8da-4516-9711-d9e1759e9a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:48.470366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:48.481834 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1113/2234] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:48.491393 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8941cf-b728-4a0e-bf97-94b55fa8c9e9 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:48.491677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8941cf-b728-4a0e-bf97-94b55fa8c9e9 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:48.491980 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8941cf-b728-4a0e-bf97-94b55fa8c9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:48.513157 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-4e8941cf-b728-4a0e-bf97-94b55fa8c9e9 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:48.514448 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-4e8941cf-b728-4a0e-bf97-94b55fa8c9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:48.523265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1122/2235] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:35:48.531163 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8ec8e-f8da-4516-9711-d9e1759e9a9a None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:48.540699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8ec8e-f8da-4516-9711-d9e1759e9a9a None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:48.547050 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9ddc2-b8f4-4119-88d5-63ee337ac048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:48.547839 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9ddc2-b8f4-4119-88d5-63ee337ac048 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:48.548183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9ddc2-b8f4-4119-88d5-63ee337ac048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:48.548526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9ddc2-b8f4-4119-88d5-63ee337ac048 admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:48.549255 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9ddc2-b8f4-4119-88d5-63ee337ac048 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:35:48.549619 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9ddc2-b8f4-4119-88d5-63ee337ac048 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:48.551336 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9ddc2-b8f4-4119-88d5-63ee337ac048 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:48.575354 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1114/2236] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:35:48.582227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8941cf-b728-4a0e-bf97-94b55fa8c9e9 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:48.594973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8941cf-b728-4a0e-bf97-94b55fa8c9e9 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:48.610295 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c545f2d-bfc3-44bc-a02b-c3cb59c3136c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:48.610295 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c545f2d-bfc3-44bc-a02b-c3cb59c3136c admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:48.610295 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c545f2d-bfc3-44bc-a02b-c3cb59c3136c admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:48.610295 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c545f2d-bfc3-44bc-a02b-c3cb59c3136c admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:48.611504 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c545f2d-bfc3-44bc-a02b-c3cb59c3136c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:48.617281 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c545f2d-bfc3-44bc-a02b-c3cb59c3136c admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:48.678277 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1123/2237] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:48 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:48.682555 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9ddc2-b8f4-4119-88d5-63ee337ac048 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:48.693244 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9ddc2-b8f4-4119-88d5-63ee337ac048 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:48.766691 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb39054-5e2f-4c2c-893d-2a1bf572a99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:48.767420 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb39054-5e2f-4c2c-893d-2a1bf572a99e admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:48.767678 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb39054-5e2f-4c2c-893d-2a1bf572a99e admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:48.767975 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb39054-5e2f-4c2c-893d-2a1bf572a99e admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:48.768650 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb39054-5e2f-4c2c-893d-2a1bf572a99e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:48.770837 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb39054-5e2f-4c2c-893d-2a1bf572a99e admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:48.793708 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1115/2238] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:48 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:48.797984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c545f2d-bfc3-44bc-a02b-c3cb59c3136c admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:48.811049 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c545f2d-bfc3-44bc-a02b-c3cb59c3136c admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:48.815867 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def7673-a4f3-437c-a964-25000b51279d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:48.816469 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def7673-a4f3-437c-a964-25000b51279d admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:48.816748 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def7673-a4f3-437c-a964-25000b51279d admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:48.817104 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def7673-a4f3-437c-a964-25000b51279d admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:48.817749 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def7673-a4f3-437c-a964-25000b51279d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:48.819279 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def7673-a4f3-437c-a964-25000b51279d admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:48.830220 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1124/2239] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:48.834197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb39054-5e2f-4c2c-893d-2a1bf572a99e admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:48.844248 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb39054-5e2f-4c2c-893d-2a1bf572a99e admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:48.848870 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527f11b-22b4-4257-b1d0-3fa33ab87cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:48.849474 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527f11b-22b4-4257-b1d0-3fa33ab87cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:48.849756 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527f11b-22b4-4257-b1d0-3fa33ab87cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:48.850032 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527f11b-22b4-4257-b1d0-3fa33ab87cd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:48.850700 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527f11b-22b4-4257-b1d0-3fa33ab87cd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:48.852340 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527f11b-22b4-4257-b1d0-3fa33ab87cd4 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:48.863021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1116/2240] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:48.867135 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def7673-a4f3-437c-a964-25000b51279d admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:48.877483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def7673-a4f3-437c-a964-25000b51279d admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:48.882083 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5741e-a80b-425a-bcb1-25b17602f713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:48.882756 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5741e-a80b-425a-bcb1-25b17602f713 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:48.883598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5741e-a80b-425a-bcb1-25b17602f713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:48.883868 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5741e-a80b-425a-bcb1-25b17602f713 admin admin] PATH_INFO: `/v3/projects/10f1f9af43e94b129a12a78888672054/users/dbd8aaf124e14138b3aa4257558ff8e6/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:48.898113 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5741e-a80b-425a-bcb1-25b17602f713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10f1f9af43e94b129a12a78888672054, user_id=dbd8aaf124e14138b3aa4257558ff8e6, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:48.900836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5741e-a80b-425a-bcb1-25b17602f713 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:48.913334 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1125/2241] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:48 2022] PUT /identity/v3/projects/10f1f9af43e94b129a12a78888672054/users/dbd8aaf124e14138b3aa4257558ff8e6/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:48.924552 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd93d2-984d-49b0-a47d-c48688ff615f None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:48.925456 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd93d2-984d-49b0-a47d-c48688ff615f None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:48.925738 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd93d2-984d-49b0-a47d-c48688ff615f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:48.956008 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-0cdd93d2-984d-49b0-a47d-c48688ff615f None None] MFA Rules not processed for user `dbd8aaf124e14138b3aa4257558ff8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:48.957219 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-0cdd93d2-984d-49b0-a47d-c48688ff615f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:49.020700 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1117/2242] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:49.025575 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5741e-a80b-425a-bcb1-25b17602f713 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:49.096111 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5741e-a80b-425a-bcb1-25b17602f713 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:49.100729 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb876a5-8fa4-42fa-b90d-137b23a6bcee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:49.101340 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb876a5-8fa4-42fa-b90d-137b23a6bcee admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:49.101659 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb876a5-8fa4-42fa-b90d-137b23a6bcee admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:49.101935 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb876a5-8fa4-42fa-b90d-137b23a6bcee admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:49.102611 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb876a5-8fa4-42fa-b90d-137b23a6bcee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:49.104781 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb876a5-8fa4-42fa-b90d-137b23a6bcee admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:49.138328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1126/2243] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:49 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:49.142641 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd93d2-984d-49b0-a47d-c48688ff615f None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:49.159791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd93d2-984d-49b0-a47d-c48688ff615f None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:49.213935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:49.222989 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfaa2c2-64bf-4272-9d5d-9db4f83a77db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:49.223971 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfaa2c2-64bf-4272-9d5d-9db4f83a77db admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:49.224421 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfaa2c2-64bf-4272-9d5d-9db4f83a77db admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:49.224869 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfaa2c2-64bf-4272-9d5d-9db4f83a77db admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:49.225733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfaa2c2-64bf-4272-9d5d-9db4f83a77db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:49.228739 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfaa2c2-64bf-4272-9d5d-9db4f83a77db admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:49.255863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1118/2244] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:49 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:49.257193 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb876a5-8fa4-42fa-b90d-137b23a6bcee admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:49.269264 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb876a5-8fa4-42fa-b90d-137b23a6bcee admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:49.273956 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b44c53a-71f6-4f01-920f-920db859f623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:49.274667 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b44c53a-71f6-4f01-920f-920db859f623 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:49.275041 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b44c53a-71f6-4f01-920f-920db859f623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:49.275221 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b44c53a-71f6-4f01-920f-920db859f623 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:49.275904 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b44c53a-71f6-4f01-920f-920db859f623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:49.277503 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b44c53a-71f6-4f01-920f-920db859f623 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:49.291856 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1127/2245] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:49.295851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfaa2c2-64bf-4272-9d5d-9db4f83a77db admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:49.307462 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfaa2c2-64bf-4272-9d5d-9db4f83a77db admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:49.312224 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96eb2f-45d3-4f66-a139-461cf86dbece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:49.312912 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96eb2f-45d3-4f66-a139-461cf86dbece admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:49.313312 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96eb2f-45d3-4f66-a139-461cf86dbece admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:49.313605 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96eb2f-45d3-4f66-a139-461cf86dbece admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:49.314338 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96eb2f-45d3-4f66-a139-461cf86dbece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:49.316262 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96eb2f-45d3-4f66-a139-461cf86dbece admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:49.329867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1119/2246] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:35:49.331194 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b44c53a-71f6-4f01-920f-920db859f623 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:49.343670 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b44c53a-71f6-4f01-920f-920db859f623 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:49.372964 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd8598-907e-4f17-be36-d002d9e81271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:49.373751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd8598-907e-4f17-be36-d002d9e81271 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:49.374110 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd8598-907e-4f17-be36-d002d9e81271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:49.374745 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd8598-907e-4f17-be36-d002d9e81271 admin admin] PATH_INFO: `/v3/projects/ee27b104875e4322b2c3804495004075/users/adfb7f81e59b429cabfc0f6abb367599/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:49.392563 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd8598-907e-4f17-be36-d002d9e81271 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee27b104875e4322b2c3804495004075, user_id=adfb7f81e59b429cabfc0f6abb367599, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:49.395216 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd8598-907e-4f17-be36-d002d9e81271 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:49.411917 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1128/2247] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:49 2022] PUT /identity/v3/projects/ee27b104875e4322b2c3804495004075/users/adfb7f81e59b429cabfc0f6abb367599/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:49.423142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa4b16-dd10-4908-820b-ee4765e036b4 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:49.423433 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa4b16-dd10-4908-820b-ee4765e036b4 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:49.423712 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa4b16-dd10-4908-820b-ee4765e036b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:49.459110 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-a7aa4b16-dd10-4908-820b-ee4765e036b4 None None] MFA Rules not processed for user `adfb7f81e59b429cabfc0f6abb367599`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:49.460317 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-a7aa4b16-dd10-4908-820b-ee4765e036b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:49.543289 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1120/2248] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:49.547909 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd8598-907e-4f17-be36-d002d9e81271 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:49.594154 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:49.618931 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd8598-907e-4f17-be36-d002d9e81271 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:49.625574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d395062-fae1-4131-9d6c-2b7ae368139a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:49.626194 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d395062-fae1-4131-9d6c-2b7ae368139a admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:49.626506 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d395062-fae1-4131-9d6c-2b7ae368139a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:49.626821 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d395062-fae1-4131-9d6c-2b7ae368139a admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:49.627470 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d395062-fae1-4131-9d6c-2b7ae368139a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:49.629379 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d395062-fae1-4131-9d6c-2b7ae368139a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:49.670752 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1129/2249] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:49 2022] POST /identity/v3/projects => generated 383 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:49.674707 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa4b16-dd10-4908-820b-ee4765e036b4 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:49.685710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa4b16-dd10-4908-820b-ee4765e036b4 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:49.744943 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a50ecc-acfe-4a34-b06c-9b568a1692b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:49.745553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a50ecc-acfe-4a34-b06c-9b568a1692b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:49.745974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a50ecc-acfe-4a34-b06c-9b568a1692b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:49.746273 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a50ecc-acfe-4a34-b06c-9b568a1692b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:49.746903 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a50ecc-acfe-4a34-b06c-9b568a1692b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:49.748839 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a50ecc-acfe-4a34-b06c-9b568a1692b1 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:49.770846 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1121/2250] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:49 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:49.775360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d395062-fae1-4131-9d6c-2b7ae368139a admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:49.793026 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d395062-fae1-4131-9d6c-2b7ae368139a admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:49.800062 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de494f3d-305a-4725-b6fe-5eb6a80b9b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:49.801077 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de494f3d-305a-4725-b6fe-5eb6a80b9b38 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:49.801670 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de494f3d-305a-4725-b6fe-5eb6a80b9b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:49.802107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de494f3d-305a-4725-b6fe-5eb6a80b9b38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:49.802846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de494f3d-305a-4725-b6fe-5eb6a80b9b38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:49.804626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de494f3d-305a-4725-b6fe-5eb6a80b9b38 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:49.815885 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1130/2251] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:49.819966 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a50ecc-acfe-4a34-b06c-9b568a1692b1 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:49.831560 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a50ecc-acfe-4a34-b06c-9b568a1692b1 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:49.837957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b071e-3233-49f3-b779-a70f470598e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:49.838664 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b071e-3233-49f3-b779-a70f470598e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:49.838996 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b071e-3233-49f3-b779-a70f470598e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:49.839274 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b071e-3233-49f3-b779-a70f470598e0 admin admin] PATH_INFO: `/v3/projects/9db4031bf080409da863959a894954ca/users/81074f50771f46219dfde939015a5699/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:49.861921 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b071e-3233-49f3-b779-a70f470598e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9db4031bf080409da863959a894954ca, user_id=81074f50771f46219dfde939015a5699, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:49.865927 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b071e-3233-49f3-b779-a70f470598e0 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:49.883635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1122/2252] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:49 2022] PUT /identity/v3/projects/9db4031bf080409da863959a894954ca/users/81074f50771f46219dfde939015a5699/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:49.885943 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de494f3d-305a-4725-b6fe-5eb6a80b9b38 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:49.896117 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de494f3d-305a-4725-b6fe-5eb6a80b9b38 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:49.960115 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b67960-211d-4fa5-af4d-419ea14cfde2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:49.961083 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b67960-211d-4fa5-af4d-419ea14cfde2 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:49.961910 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b67960-211d-4fa5-af4d-419ea14cfde2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:49.962187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b67960-211d-4fa5-af4d-419ea14cfde2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:49.962869 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b67960-211d-4fa5-af4d-419ea14cfde2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:49.964640 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b67960-211d-4fa5-af4d-419ea14cfde2 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:49.974597 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1131/2253] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:49.979568 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b071e-3233-49f3-b779-a70f470598e0 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:49.990007 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b071e-3233-49f3-b779-a70f470598e0 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:49.994789 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4e96f-8cfe-4760-b154-ffd4dd91b81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:49.995435 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4e96f-8cfe-4760-b154-ffd4dd91b81e admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:49.995714 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4e96f-8cfe-4760-b154-ffd4dd91b81e admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:49.995986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4e96f-8cfe-4760-b154-ffd4dd91b81e admin admin] PATH_INFO: `/v3/projects/9db4031bf080409da863959a894954ca/users/81074f50771f46219dfde939015a5699/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:49.997936 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f4e96f-8cfe-4760-b154-ffd4dd91b81e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9db4031bf080409da863959a894954ca, user_id=81074f50771f46219dfde939015a5699, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:50.000644 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f4e96f-8cfe-4760-b154-ffd4dd91b81e admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:50.011647 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1123/2254] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:49 2022] PUT /identity/v3/projects/9db4031bf080409da863959a894954ca/users/81074f50771f46219dfde939015a5699/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:50.023330 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc516859-a79b-403c-a65a-1733f85dbd42 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:50.024630 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc516859-a79b-403c-a65a-1733f85dbd42 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:50.025253 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc516859-a79b-403c-a65a-1733f85dbd42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:50.057642 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-cc516859-a79b-403c-a65a-1733f85dbd42 None None] MFA Rules not processed for user `81074f50771f46219dfde939015a5699`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:50.058884 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-cc516859-a79b-403c-a65a-1733f85dbd42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:50.140474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1132/2255] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:50 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:50.152741 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4e96f-8cfe-4760-b154-ffd4dd91b81e admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:50.158724 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-e0f4e96f-8cfe-4760-b154-ffd4dd91b81e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:50.179732 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:50.220876 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4e96f-8cfe-4760-b154-ffd4dd91b81e admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:50.224776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743a35f-7cc0-48a0-bcf0-34e60aab68b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:50.225527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743a35f-7cc0-48a0-bcf0-34e60aab68b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:50.225695 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743a35f-7cc0-48a0-bcf0-34e60aab68b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:50.225968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743a35f-7cc0-48a0-bcf0-34e60aab68b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:50.286400 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743a35f-7cc0-48a0-bcf0-34e60aab68b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:50.287859 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743a35f-7cc0-48a0-bcf0-34e60aab68b0 service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:50.298781 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1124/2256] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:50.647993 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc516859-a79b-403c-a65a-1733f85dbd42 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:50.651763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-cc516859-a79b-403c-a65a-1733f85dbd42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:50.731080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:50.756266 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc516859-a79b-403c-a65a-1733f85dbd42 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:50.760813 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66692dd9-4f0a-4c50-982c-5554e377c655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:50.761452 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66692dd9-4f0a-4c50-982c-5554e377c655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:50.761734 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66692dd9-4f0a-4c50-982c-5554e377c655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:50.762012 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66692dd9-4f0a-4c50-982c-5554e377c655 admin admin] PATH_INFO: `/v3/users/439b001718a047528f2f670c39b3e87d` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:50.763176 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66692dd9-4f0a-4c50-982c-5554e377c655 admin admin] RBAC: Authorizing `identity:delete_user(user_id=439b001718a047528f2f670c39b3e87d)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:50.765084 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66692dd9-4f0a-4c50-982c-5554e377c655 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:50.814846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-66692dd9-4f0a-4c50-982c-5554e377c655 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '439b001718a047528f2f670c39b3e87d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:50.823282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-66692dd9-4f0a-4c50-982c-5554e377c655 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '439b001718a047528f2f670c39b3e87d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:50.823924 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-66692dd9-4f0a-4c50-982c-5554e377c655 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '439b001718a047528f2f670c39b3e87d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:50.836839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-66692dd9-4f0a-4c50-982c-5554e377c655 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '439b001718a047528f2f670c39b3e87d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:50.837296 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-66692dd9-4f0a-4c50-982c-5554e377c655 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '439b001718a047528f2f670c39b3e87d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:50.856382 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1133/2257] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:50 2022] DELETE /identity/v3/users/439b001718a047528f2f670c39b3e87d => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:50.860880 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dec205-a064-46b9-9cd8-97a3c7e45d32 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:50.861337 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dec205-a064-46b9-9cd8-97a3c7e45d32 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:50.861427 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dec205-a064-46b9-9cd8-97a3c7e45d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:50.881912 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-56dec205-a064-46b9-9cd8-97a3c7e45d32 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:50.883126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-56dec205-a064-46b9-9cd8-97a3c7e45d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:50.938667 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66692dd9-4f0a-4c50-982c-5554e377c655 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:50.940174 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-66692dd9-4f0a-4c50-982c-5554e377c655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:50.943039 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1125/2258] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:50.952687 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dec205-a064-46b9-9cd8-97a3c7e45d32 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:50.954125 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-56dec205-a064-46b9-9cd8-97a3c7e45d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:50.957029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66692dd9-4f0a-4c50-982c-5554e377c655 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:50.962448 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7533b-d419-4c84-99dc-e6b27299467f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:50.963097 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7533b-d419-4c84-99dc-e6b27299467f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:50.963382 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7533b-d419-4c84-99dc-e6b27299467f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:50.963662 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7533b-d419-4c84-99dc-e6b27299467f admin admin] PATH_INFO: `/v3/users/b3f9e14f3b604a9bbba0f75620f28555` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:50.964803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a7533b-d419-4c84-99dc-e6b27299467f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3f9e14f3b604a9bbba0f75620f28555)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:50.966782 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a7533b-d419-4c84-99dc-e6b27299467f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:51.013984 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dec205-a064-46b9-9cd8-97a3c7e45d32 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:51.015834 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-16a7533b-d419-4c84-99dc-e6b27299467f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3f9e14f3b604a9bbba0f75620f28555'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:51.017913 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ee1206-65da-4892-89df-25190cc8557a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:51.018554 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee1206-65da-4892-89df-25190cc8557a service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:51.018833 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee1206-65da-4892-89df-25190cc8557a service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:51.019167 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee1206-65da-4892-89df-25190cc8557a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:51.023201 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-16a7533b-d419-4c84-99dc-e6b27299467f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3f9e14f3b604a9bbba0f75620f28555'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:51.024028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-16a7533b-d419-4c84-99dc-e6b27299467f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3f9e14f3b604a9bbba0f75620f28555'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:51.028752 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ee1206-65da-4892-89df-25190cc8557a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:51.030204 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ee1206-65da-4892-89df-25190cc8557a service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:51.036240 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-16a7533b-d419-4c84-99dc-e6b27299467f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3f9e14f3b604a9bbba0f75620f28555'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:51.036932 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-16a7533b-d419-4c84-99dc-e6b27299467f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3f9e14f3b604a9bbba0f75620f28555'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:51.040791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1126/2259] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:35:51.049428 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1134/2260] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:50 2022] DELETE /identity/v3/users/b3f9e14f3b604a9bbba0f75620f28555 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:51.054084 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb11d0b-6d67-4264-a8fd-6a3d1d5590c0 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:51.054413 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb11d0b-6d67-4264-a8fd-6a3d1d5590c0 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:51.055250 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb11d0b-6d67-4264-a8fd-6a3d1d5590c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:51.073990 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-3fb11d0b-6d67-4264-a8fd-6a3d1d5590c0 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:51.075261 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-3fb11d0b-6d67-4264-a8fd-6a3d1d5590c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:51.083549 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:51.137857 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1127/2261] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:51.147746 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7533b-d419-4c84-99dc-e6b27299467f admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:51.149929 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-16a7533b-d419-4c84-99dc-e6b27299467f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:51.184541 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:51.234600 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7533b-d419-4c84-99dc-e6b27299467f admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:51.239399 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d5427d-5738-4e9d-a349-5c9f7dc9a6f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:51.240250 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5427d-5738-4e9d-a349-5c9f7dc9a6f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:51.240492 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5427d-5738-4e9d-a349-5c9f7dc9a6f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:51.240767 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5427d-5738-4e9d-a349-5c9f7dc9a6f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:51.253490 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d5427d-5738-4e9d-a349-5c9f7dc9a6f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:51.255026 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d5427d-5738-4e9d-a349-5c9f7dc9a6f5 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:51.267516 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1135/2262] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:35:51.370364 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb11d0b-6d67-4264-a8fd-6a3d1d5590c0 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:51.374226 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-3fb11d0b-6d67-4264-a8fd-6a3d1d5590c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:51.390592 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb11d0b-6d67-4264-a8fd-6a3d1d5590c0 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:51.399442 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f20625-bddf-4919-beda-f1065c75713a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:51.400166 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f20625-bddf-4919-beda-f1065c75713a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:51.400593 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f20625-bddf-4919-beda-f1065c75713a admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:51.400965 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f20625-bddf-4919-beda-f1065c75713a admin admin] PATH_INFO: `/v3/projects/a9be0a80b8664b3da5467a729ba4061c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:51.402202 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f20625-bddf-4919-beda-f1065c75713a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9be0a80b8664b3da5467a729ba4061c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:51.404495 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f20625-bddf-4919-beda-f1065c75713a admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:51.464222 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-d1f20625-bddf-4919-beda-f1065c75713a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9be0a80b8664b3da5467a729ba4061c', 'request_id': 'req-d1f20625-bddf-4919-beda-f1065c75713a'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:51.472539 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-d1f20625-bddf-4919-beda-f1065c75713a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9be0a80b8664b3da5467a729ba4061c', 'request_id': 'req-d1f20625-bddf-4919-beda-f1065c75713a'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:51.480312 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-d1f20625-bddf-4919-beda-f1065c75713a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9be0a80b8664b3da5467a729ba4061c', 'request_id': 'req-d1f20625-bddf-4919-beda-f1065c75713a'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:51.493498 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-d1f20625-bddf-4919-beda-f1065c75713a admin admin] The token cache is being invalidate because project a9be0a80b8664b3da5467a729ba4061c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:51.493789 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-d1f20625-bddf-4919-beda-f1065c75713a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:51.494473 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-d1f20625-bddf-4919-beda-f1065c75713a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:51.496505 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1128/2263] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:51 2022] DELETE /identity/v3/projects/a9be0a80b8664b3da5467a729ba4061c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:51.501291 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc3ae2-6d52-4908-968f-0dbe4c22858e None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:51.501585 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc3ae2-6d52-4908-968f-0dbe4c22858e None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:51.501867 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc3ae2-6d52-4908-968f-0dbe4c22858e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:51.529100 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-75fc3ae2-6d52-4908-968f-0dbe4c22858e None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:51.530503 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-75fc3ae2-6d52-4908-968f-0dbe4c22858e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:51.605749 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f20625-bddf-4919-beda-f1065c75713a admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:51.607449 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-d1f20625-bddf-4919-beda-f1065c75713a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:51.627051 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1136/2264] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:51.639323 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993f1d2-1603-4f04-931b-8ab81a09d54d None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:51.639833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993f1d2-1603-4f04-931b-8ab81a09d54d None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:51.639917 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993f1d2-1603-4f04-931b-8ab81a09d54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:51.671873 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-c993f1d2-1603-4f04-931b-8ab81a09d54d None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:51.673341 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c993f1d2-1603-4f04-931b-8ab81a09d54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:51.690308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1137/2265] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:35:51.691899 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f20625-bddf-4919-beda-f1065c75713a admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:51.693660 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993f1d2-1603-4f04-931b-8ab81a09d54d None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:51.701577 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc8f9a0-90b9-45c9-a73b-2a5132c34bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:51.702697 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc8f9a0-90b9-45c9-a73b-2a5132c34bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:51.703217 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc8f9a0-90b9-45c9-a73b-2a5132c34bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:51.703838 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc8f9a0-90b9-45c9-a73b-2a5132c34bf1 admin admin] PATH_INFO: `/v3/projects/e6395f60c0df449d8b2afe28d84a4851` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:51.705169 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993f1d2-1603-4f04-931b-8ab81a09d54d None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:51.705930 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc8f9a0-90b9-45c9-a73b-2a5132c34bf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6395f60c0df449d8b2afe28d84a4851)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:51.708570 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc8f9a0-90b9-45c9-a73b-2a5132c34bf1 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:51.711062 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d2906-dc6e-4d1d-a102-8aa07d544a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:51.712768 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d2906-dc6e-4d1d-a102-8aa07d544a42 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:51.713230 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d2906-dc6e-4d1d-a102-8aa07d544a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:51.713846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d2906-dc6e-4d1d-a102-8aa07d544a42 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:51.714853 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d2906-dc6e-4d1d-a102-8aa07d544a42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:35:51.715338 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d2906-dc6e-4d1d-a102-8aa07d544a42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:51.717080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d2906-dc6e-4d1d-a102-8aa07d544a42 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:51.731251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1138/2266] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:35:51.735982 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d2906-dc6e-4d1d-a102-8aa07d544a42 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:51.753751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d2906-dc6e-4d1d-a102-8aa07d544a42 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:51.760912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3132ca-931e-4844-ab37-089889b29834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:51.762838 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3132ca-931e-4844-ab37-089889b29834 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:51.763157 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3132ca-931e-4844-ab37-089889b29834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:51.763666 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3132ca-931e-4844-ab37-089889b29834 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:51.765092 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3132ca-931e-4844-ab37-089889b29834 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:51.783991 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3132ca-931e-4844-ab37-089889b29834 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:51.785433 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:51.804394 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-2bc8f9a0-90b9-45c9-a73b-2a5132c34bf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6395f60c0df449d8b2afe28d84a4851', 'request_id': 'req-2bc8f9a0-90b9-45c9-a73b-2a5132c34bf1'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:51.826767 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-2bc8f9a0-90b9-45c9-a73b-2a5132c34bf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6395f60c0df449d8b2afe28d84a4851', 'request_id': 'req-2bc8f9a0-90b9-45c9-a73b-2a5132c34bf1'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:51.842275 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1139/2267] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:51 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:51.845437 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-2bc8f9a0-90b9-45c9-a73b-2a5132c34bf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6395f60c0df449d8b2afe28d84a4851', 'request_id': 'req-2bc8f9a0-90b9-45c9-a73b-2a5132c34bf1'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:51.855754 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3132ca-931e-4844-ab37-089889b29834 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:51.861963 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-2bc8f9a0-90b9-45c9-a73b-2a5132c34bf1 admin admin] The token cache is being invalidate because project e6395f60c0df449d8b2afe28d84a4851 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:51.862529 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-2bc8f9a0-90b9-45c9-a73b-2a5132c34bf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:51.863209 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-2bc8f9a0-90b9-45c9-a73b-2a5132c34bf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:51.866066 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1129/2268] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:51 2022] DELETE /identity/v3/projects/e6395f60c0df449d8b2afe28d84a4851 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:51.871357 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc8f9a0-90b9-45c9-a73b-2a5132c34bf1 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:51.873627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-2bc8f9a0-90b9-45c9-a73b-2a5132c34bf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:51.952429 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3132ca-931e-4844-ab37-089889b29834 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:51.959184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227ea5d2-aed3-412a-9635-225e577ca551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:51.960197 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ea5d2-aed3-412a-9635-225e577ca551 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:51.960652 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ea5d2-aed3-412a-9635-225e577ca551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:51.961100 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ea5d2-aed3-412a-9635-225e577ca551 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:51.962001 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227ea5d2-aed3-412a-9635-225e577ca551 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:51.976611 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc8f9a0-90b9-45c9-a73b-2a5132c34bf1 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:51.976728 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227ea5d2-aed3-412a-9635-225e577ca551 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:51.987414 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b242b22-9f0c-40fd-b429-6e0d60e92e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:51.989380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b242b22-9f0c-40fd-b429-6e0d60e92e80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:51.989680 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b242b22-9f0c-40fd-b429-6e0d60e92e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:51.990042 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b242b22-9f0c-40fd-b429-6e0d60e92e80 admin admin] PATH_INFO: `/v3/users/d098c0dee61b450780beb848ce176958` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:51.995605 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b242b22-9f0c-40fd-b429-6e0d60e92e80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d098c0dee61b450780beb848ce176958)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:52.000507 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b242b22-9f0c-40fd-b429-6e0d60e92e80 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:52.002588 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1140/2269] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:51 2022] POST /identity/v3/users => generated 293 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:52.006507 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227ea5d2-aed3-412a-9635-225e577ca551 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:52.010061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-227ea5d2-aed3-412a-9635-225e577ca551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:52.056715 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227ea5d2-aed3-412a-9635-225e577ca551 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:52.063666 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1b3d0-24b3-46b2-89ff-7f52e7d7a0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:52.065303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1b3d0-24b3-46b2-89ff-7f52e7d7a0ef admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:52.065807 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1b3d0-24b3-46b2-89ff-7f52e7d7a0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:52.066413 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1b3d0-24b3-46b2-89ff-7f52e7d7a0ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:52.075735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c1b3d0-24b3-46b2-89ff-7f52e7d7a0ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:52.078265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c1b3d0-24b3-46b2-89ff-7f52e7d7a0ef admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:52.083247 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-7b242b22-9f0c-40fd-b429-6e0d60e92e80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd098c0dee61b450780beb848ce176958'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:52.084737 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-7b242b22-9f0c-40fd-b429-6e0d60e92e80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd098c0dee61b450780beb848ce176958'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:52.104043 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1141/2270] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:52.106480 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-7b242b22-9f0c-40fd-b429-6e0d60e92e80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd098c0dee61b450780beb848ce176958'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:52.107553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-7b242b22-9f0c-40fd-b429-6e0d60e92e80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd098c0dee61b450780beb848ce176958'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:52.110408 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1b3d0-24b3-46b2-89ff-7f52e7d7a0ef admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:52.113944 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-39c1b3d0-24b3-46b2-89ff-7f52e7d7a0ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:52.127740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-7b242b22-9f0c-40fd-b429-6e0d60e92e80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd098c0dee61b450780beb848ce176958'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:52.128557 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:52.155390 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1130/2271] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:51 2022] DELETE /identity/v3/users/d098c0dee61b450780beb848ce176958 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:52.156564 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b242b22-9f0c-40fd-b429-6e0d60e92e80 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:52.159315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-7b242b22-9f0c-40fd-b429-6e0d60e92e80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:52.208138 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1b3d0-24b3-46b2-89ff-7f52e7d7a0ef admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:52.275918 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:52.280005 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa89951-63c7-4ffa-a8df-51a80c1b05b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:52.280757 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa89951-63c7-4ffa-a8df-51a80c1b05b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:52.281143 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa89951-63c7-4ffa-a8df-51a80c1b05b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:52.281741 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa89951-63c7-4ffa-a8df-51a80c1b05b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:52.285728 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-baa89951-63c7-4ffa-a8df-51a80c1b05b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:52.298021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b242b22-9f0c-40fd-b429-6e0d60e92e80 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:52.305300 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a2f59-0c21-46e7-b8a5-d3ea16c9e433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:52.307105 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a2f59-0c21-46e7-b8a5-d3ea16c9e433 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:52.307747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a2f59-0c21-46e7-b8a5-d3ea16c9e433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:52.308720 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a2f59-0c21-46e7-b8a5-d3ea16c9e433 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:52.309630 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a2f59-0c21-46e7-b8a5-d3ea16c9e433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:52.312052 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a2f59-0c21-46e7-b8a5-d3ea16c9e433 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:52.328318 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1131/2272] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:52.333671 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a2f59-0c21-46e7-b8a5-d3ea16c9e433 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:52.357306 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a2f59-0c21-46e7-b8a5-d3ea16c9e433 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:52.371659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b033c08-31da-41bc-8c55-a3519b2021ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:52.372531 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b033c08-31da-41bc-8c55-a3519b2021ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:52.372817 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b033c08-31da-41bc-8c55-a3519b2021ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:52.373129 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b033c08-31da-41bc-8c55-a3519b2021ad admin admin] PATH_INFO: `/v3/projects/d1acfef541384c44bd383879051f20d0/users/3a14e6faf6194244810ef5d405a6da8d/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:52.387671 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa89951-63c7-4ffa-a8df-51a80c1b05b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:52.390752 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa89951-63c7-4ffa-a8df-51a80c1b05b9 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:52.394383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b033c08-31da-41bc-8c55-a3519b2021ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1acfef541384c44bd383879051f20d0, user_id=3a14e6faf6194244810ef5d405a6da8d, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:52.397301 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b033c08-31da-41bc-8c55-a3519b2021ad admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:52.416032 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1132/2273] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:52 2022] PUT /identity/v3/projects/d1acfef541384c44bd383879051f20d0/users/3a14e6faf6194244810ef5d405a6da8d/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:52.418382 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1142/2274] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:52.428036 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f582c-69c8-42ba-9f99-df278a9c84d3 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:52.428606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f582c-69c8-42ba-9f99-df278a9c84d3 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:52.429080 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f582c-69c8-42ba-9f99-df278a9c84d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:52.508844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-563f582c-69c8-42ba-9f99-df278a9c84d3 None None] MFA Rules not processed for user `3a14e6faf6194244810ef5d405a6da8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:52.511203 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-563f582c-69c8-42ba-9f99-df278a9c84d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:52.528992 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa89951-63c7-4ffa-a8df-51a80c1b05b9 service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:52.531068 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-baa89951-63c7-4ffa-a8df-51a80c1b05b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:52.569597 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa89951-63c7-4ffa-a8df-51a80c1b05b9 service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:52.639399 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1133/2275] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:52.642956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f582c-69c8-42ba-9f99-df278a9c84d3 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:52.692563 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c38c70b-6aad-4c7a-a2cc-15889ecd8290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:52.693348 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38c70b-6aad-4c7a-a2cc-15889ecd8290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:52.693631 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38c70b-6aad-4c7a-a2cc-15889ecd8290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:52.694107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38c70b-6aad-4c7a-a2cc-15889ecd8290 admin admin] PATH_INFO: `/v3/projects/372b9ecc883f4bd4a02e544814cb800b` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:52.696077 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c38c70b-6aad-4c7a-a2cc-15889ecd8290 admin admin] RBAC: Authorizing `identity:delete_project(project_id=372b9ecc883f4bd4a02e544814cb800b)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:52.699058 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c38c70b-6aad-4c7a-a2cc-15889ecd8290 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:52.734846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:52.767516 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5c38c70b-6aad-4c7a-a2cc-15889ecd8290 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '372b9ecc883f4bd4a02e544814cb800b', 'request_id': 'req-5c38c70b-6aad-4c7a-a2cc-15889ecd8290'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:52.769035 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f582c-69c8-42ba-9f99-df278a9c84d3 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:52.776572 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5c38c70b-6aad-4c7a-a2cc-15889ecd8290 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '372b9ecc883f4bd4a02e544814cb800b', 'request_id': 'req-5c38c70b-6aad-4c7a-a2cc-15889ecd8290'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:52.788377 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5c38c70b-6aad-4c7a-a2cc-15889ecd8290 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '372b9ecc883f4bd4a02e544814cb800b', 'request_id': 'req-5c38c70b-6aad-4c7a-a2cc-15889ecd8290'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:52.801847 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5c38c70b-6aad-4c7a-a2cc-15889ecd8290 admin admin] The token cache is being invalidate because project 372b9ecc883f4bd4a02e544814cb800b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:52.802471 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5c38c70b-6aad-4c7a-a2cc-15889ecd8290 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:52.803247 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5c38c70b-6aad-4c7a-a2cc-15889ecd8290 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:52.805120 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1143/2276] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:52 2022] DELETE /identity/v3/projects/372b9ecc883f4bd4a02e544814cb800b => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:52.810596 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff0bf3-5ba4-42b2-a994-f7d8e8f8e4ce None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:52.811573 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff0bf3-5ba4-42b2-a994-f7d8e8f8e4ce None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:52.811958 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff0bf3-5ba4-42b2-a994-f7d8e8f8e4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:52.839316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-7cff0bf3-5ba4-42b2-a994-f7d8e8f8e4ce None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:52.843981 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7cff0bf3-5ba4-42b2-a994-f7d8e8f8e4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:52.854386 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:52.868618 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaadd0f-c3df-45cf-b7dd-8cbe4e1fffc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:52.869535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaadd0f-c3df-45cf-b7dd-8cbe4e1fffc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:52.870606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaadd0f-c3df-45cf-b7dd-8cbe4e1fffc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:52.870988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaadd0f-c3df-45cf-b7dd-8cbe4e1fffc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:52.872094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaadd0f-c3df-45cf-b7dd-8cbe4e1fffc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:52.875065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaadd0f-c3df-45cf-b7dd-8cbe4e1fffc4 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:52.907957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1134/2277] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:52 2022] POST /identity/v3/projects => generated 383 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:52.912286 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaadd0f-c3df-45cf-b7dd-8cbe4e1fffc4 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:52.916220 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-efaadd0f-c3df-45cf-b7dd-8cbe4e1fffc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:52.943868 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1144/2278] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:35:52.947437 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4642ad-983c-4217-9df1-a5c81e669825 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:52.947764 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4642ad-983c-4217-9df1-a5c81e669825 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:52.947994 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4642ad-983c-4217-9df1-a5c81e669825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:52.979314 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-ae4642ad-983c-4217-9df1-a5c81e669825 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:52.980541 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ae4642ad-983c-4217-9df1-a5c81e669825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:53.036995 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaadd0f-c3df-45cf-b7dd-8cbe4e1fffc4 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:53.036995 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ff1b4-a1c7-4428-a116-d7b188a6b48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:53.036995 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ff1b4-a1c7-4428-a116-d7b188a6b48c admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:53.036995 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ff1b4-a1c7-4428-a116-d7b188a6b48c admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:53.037422 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ff1b4-a1c7-4428-a116-d7b188a6b48c admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:53.037422 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ff1b4-a1c7-4428-a116-d7b188a6b48c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:53.037422 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ff1b4-a1c7-4428-a116-d7b188a6b48c admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:53.074066 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1135/2279] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:52 2022] POST /identity/v3/users => generated 292 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:53.079235 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ff1b4-a1c7-4428-a116-d7b188a6b48c admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:53.090774 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1145/2280] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:35:53.094735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4642ad-983c-4217-9df1-a5c81e669825 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:53.100832 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ff1b4-a1c7-4428-a116-d7b188a6b48c admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:53.127898 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ef845-de2d-4821-a934-83b5d9459f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:53.130276 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4642ad-983c-4217-9df1-a5c81e669825 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:53.131060 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ef845-de2d-4821-a934-83b5d9459f56 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:53.131568 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ef845-de2d-4821-a934-83b5d9459f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:53.131841 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ef845-de2d-4821-a934-83b5d9459f56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:53.132500 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50ef845-de2d-4821-a934-83b5d9459f56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:53.135240 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50ef845-de2d-4821-a934-83b5d9459f56 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:53.156056 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd93268-6062-4e11-b9f9-03f5702a89c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:53.156898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd93268-6062-4e11-b9f9-03f5702a89c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:53.157185 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd93268-6062-4e11-b9f9-03f5702a89c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:53.157504 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd93268-6062-4e11-b9f9-03f5702a89c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:53.158104 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd93268-6062-4e11-b9f9-03f5702a89c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:35:53.158547 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd93268-6062-4e11-b9f9-03f5702a89c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:53.160123 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd93268-6062-4e11-b9f9-03f5702a89c8 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:53.169444 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1136/2281] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:53.173078 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ef845-de2d-4821-a934-83b5d9459f56 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:53.186323 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1146/2282] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:35:53.191125 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd93268-6062-4e11-b9f9-03f5702a89c8 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:53.204189 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd93268-6062-4e11-b9f9-03f5702a89c8 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:53.204867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ef845-de2d-4821-a934-83b5d9459f56 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:53.209625 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1967da-7c43-4718-a46e-4a392f699f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:53.210449 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1967da-7c43-4718-a46e-4a392f699f2a admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:53.211444 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1967da-7c43-4718-a46e-4a392f699f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:53.212085 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1967da-7c43-4718-a46e-4a392f699f2a admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:53.213024 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1967da-7c43-4718-a46e-4a392f699f2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:53.214513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f7471-ebca-45c6-82ac-361694c639b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:53.215199 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f7471-ebca-45c6-82ac-361694c639b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:53.215688 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f7471-ebca-45c6-82ac-361694c639b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:53.215901 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1967da-7c43-4718-a46e-4a392f699f2a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:53.216017 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f7471-ebca-45c6-82ac-361694c639b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:53.230815 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f7471-ebca-45c6-82ac-361694c639b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:53.232523 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f7471-ebca-45c6-82ac-361694c639b6 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:53.291597 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1137/2283] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:35:53.294224 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f7471-ebca-45c6-82ac-361694c639b6 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:53.310372 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1147/2284] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:53 2022] POST /identity/v3/projects => generated 361 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:53.312739 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1967da-7c43-4718-a46e-4a392f699f2a admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:53.326300 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f7471-ebca-45c6-82ac-361694c639b6 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:53.359476 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:53.466569 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0887a-3d2c-4074-87d3-5d45c75c72ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:53.467310 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0887a-3d2c-4074-87d3-5d45c75c72ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:53.467559 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0887a-3d2c-4074-87d3-5d45c75c72ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:53.467835 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0887a-3d2c-4074-87d3-5d45c75c72ab admin admin] PATH_INFO: `/v3/projects/96d06633b175479d8b0479a72affdcd7/users/6816abc4bfdf41d39ef5a93f3fe0ceb0/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:53.471002 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1967da-7c43-4718-a46e-4a392f699f2a admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:53.481478 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c79e50-6ab7-4c44-8e7b-60861b2daea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:53.482473 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c79e50-6ab7-4c44-8e7b-60861b2daea9 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:53.482954 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c79e50-6ab7-4c44-8e7b-60861b2daea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:53.483234 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c79e50-6ab7-4c44-8e7b-60861b2daea9 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:53.483928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c79e50-6ab7-4c44-8e7b-60861b2daea9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:53.485956 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c79e50-6ab7-4c44-8e7b-60861b2daea9 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:53.498018 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe0887a-3d2c-4074-87d3-5d45c75c72ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=96d06633b175479d8b0479a72affdcd7, user_id=6816abc4bfdf41d39ef5a93f3fe0ceb0, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:53.500753 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe0887a-3d2c-4074-87d3-5d45c75c72ab admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:53.514294 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1148/2285] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:53 2022] POST /identity/v3/users => generated 281 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:53.514294 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c79e50-6ab7-4c44-8e7b-60861b2daea9 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:53.514294 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b6c79e50-6ab7-4c44-8e7b-60861b2daea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:53.525005 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1138/2286] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:53 2022] PUT /identity/v3/projects/96d06633b175479d8b0479a72affdcd7/users/6816abc4bfdf41d39ef5a93f3fe0ceb0/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:53.525666 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0887a-3d2c-4074-87d3-5d45c75c72ab admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:53.569100 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:53.610913 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c79e50-6ab7-4c44-8e7b-60861b2daea9 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:53.614011 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0887a-3d2c-4074-87d3-5d45c75c72ab admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:53.619616 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c2b14-4818-48ef-bc2d-63aedc4a0982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:53.620242 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c2b14-4818-48ef-bc2d-63aedc4a0982 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:53.620526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c2b14-4818-48ef-bc2d-63aedc4a0982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:53.620832 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c2b14-4818-48ef-bc2d-63aedc4a0982 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:53.621510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582c2b14-4818-48ef-bc2d-63aedc4a0982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:53.623226 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582c2b14-4818-48ef-bc2d-63aedc4a0982 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:53.634351 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1139/2287] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:53.636925 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05c5b9-51d5-4a23-bd7a-a398e7cc0faa None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:53.636925 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05c5b9-51d5-4a23-bd7a-a398e7cc0faa None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:53.636925 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05c5b9-51d5-4a23-bd7a-a398e7cc0faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:53.662924 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b850e8aa-e8e7-45f0-8cc4-02e36b197edf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:53.663549 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850e8aa-e8e7-45f0-8cc4-02e36b197edf service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:53.663832 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850e8aa-e8e7-45f0-8cc4-02e36b197edf service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:53.664622 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850e8aa-e8e7-45f0-8cc4-02e36b197edf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:53.666713 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b850e8aa-e8e7-45f0-8cc4-02e36b197edf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:53.667020 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-dc05c5b9-51d5-4a23-bd7a-a398e7cc0faa None None] MFA Rules not processed for user `6816abc4bfdf41d39ef5a93f3fe0ceb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:53.668208 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-dc05c5b9-51d5-4a23-bd7a-a398e7cc0faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:53.735262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b850e8aa-e8e7-45f0-8cc4-02e36b197edf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:53.736788 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b850e8aa-e8e7-45f0-8cc4-02e36b197edf service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:53.747910 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1140/2288] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:53.747910 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc05c5b9-51d5-4a23-bd7a-a398e7cc0faa None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:53.748171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1149/2289] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:53 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:53.748883 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b850e8aa-e8e7-45f0-8cc4-02e36b197edf service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:53.758320 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc05c5b9-51d5-4a23-bd7a-a398e7cc0faa None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:53.759924 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b850e8aa-e8e7-45f0-8cc4-02e36b197edf service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:53.764867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f075f3-e430-4df9-a3d7-9e596cfe0497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:53.765086 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca37b99-d230-45be-a429-e7b023d9b90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:53.765484 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f075f3-e430-4df9-a3d7-9e596cfe0497 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:53.765661 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca37b99-d230-45be-a429-e7b023d9b90a admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:53.765778 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f075f3-e430-4df9-a3d7-9e596cfe0497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:53.765929 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca37b99-d230-45be-a429-e7b023d9b90a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:53.766026 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f075f3-e430-4df9-a3d7-9e596cfe0497 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:53.766194 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca37b99-d230-45be-a429-e7b023d9b90a admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:53.766704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f075f3-e430-4df9-a3d7-9e596cfe0497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:53.766862 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca37b99-d230-45be-a429-e7b023d9b90a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:53.768445 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f075f3-e430-4df9-a3d7-9e596cfe0497 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:53.768748 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca37b99-d230-45be-a429-e7b023d9b90a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:53.778793 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:53.781693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1141/2290] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:53.786125 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f075f3-e430-4df9-a3d7-9e596cfe0497 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:53.797891 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f075f3-e430-4df9-a3d7-9e596cfe0497 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:53.803622 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00f3c4-906e-4b91-8c22-0f4b1528ad5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:53.804279 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00f3c4-906e-4b91-8c22-0f4b1528ad5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:53.805376 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00f3c4-906e-4b91-8c22-0f4b1528ad5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:53.805714 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00f3c4-906e-4b91-8c22-0f4b1528ad5a admin admin] PATH_INFO: `/v3/projects/a5781da6630a4238b361fcb1754fd237/users/1a7e3b35d8cb4df88d620c5db7fb426b/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:53.810870 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1150/2291] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:53 2022] POST /identity/v3/projects => generated 381 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:53.814848 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca37b99-d230-45be-a429-e7b023d9b90a admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:53.822345 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f00f3c4-906e-4b91-8c22-0f4b1528ad5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5781da6630a4238b361fcb1754fd237, user_id=1a7e3b35d8cb4df88d620c5db7fb426b, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:53.825521 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f00f3c4-906e-4b91-8c22-0f4b1528ad5a admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:53.840915 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1142/2292] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:53 2022] PUT /identity/v3/projects/a5781da6630a4238b361fcb1754fd237/users/1a7e3b35d8cb4df88d620c5db7fb426b/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:53.852332 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d71c47b-dc03-4ef9-acfb-9398f4cb9e44 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:53.853008 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d71c47b-dc03-4ef9-acfb-9398f4cb9e44 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:53.853008 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d71c47b-dc03-4ef9-acfb-9398f4cb9e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:53.884146 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-1d71c47b-dc03-4ef9-acfb-9398f4cb9e44 None None] MFA Rules not processed for user `1a7e3b35d8cb4df88d620c5db7fb426b`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:53.885467 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-1d71c47b-dc03-4ef9-acfb-9398f4cb9e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:53.891743 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca37b99-d230-45be-a429-e7b023d9b90a admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:53.949926 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1143/2293] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:35:53.952356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f002767-7e2d-44f7-9c11-cdcccc6861b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:53.953356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f002767-7e2d-44f7-9c11-cdcccc6861b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:53.953917 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f002767-7e2d-44f7-9c11-cdcccc6861b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:53.954257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f002767-7e2d-44f7-9c11-cdcccc6861b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:53.957333 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f002767-7e2d-44f7-9c11-cdcccc6861b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:53.957333 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f002767-7e2d-44f7-9c11-cdcccc6861b7 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:53.984576 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1151/2294] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:53 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:53.989334 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d71c47b-dc03-4ef9-acfb-9398f4cb9e44 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:53.990464 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f002767-7e2d-44f7-9c11-cdcccc6861b7 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:53.992843 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7f002767-7e2d-44f7-9c11-cdcccc6861b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:54.003321 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d71c47b-dc03-4ef9-acfb-9398f4cb9e44 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:54.008364 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee767c4-a8be-4883-a17f-d075b050b03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:54.009155 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee767c4-a8be-4883-a17f-d075b050b03c admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:54.009440 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee767c4-a8be-4883-a17f-d075b050b03c admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:54.009715 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee767c4-a8be-4883-a17f-d075b050b03c admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:54.010538 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee767c4-a8be-4883-a17f-d075b050b03c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:54.012210 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee767c4-a8be-4883-a17f-d075b050b03c admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:54.017705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:54.023649 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1144/2295] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:54.035967 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee767c4-a8be-4883-a17f-d075b050b03c admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:54.049578 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee767c4-a8be-4883-a17f-d075b050b03c admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:54.058460 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838e067-97ff-4c4c-852e-f462f4ed10de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:54.059133 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838e067-97ff-4c4c-852e-f462f4ed10de admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:54.059419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838e067-97ff-4c4c-852e-f462f4ed10de admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:54.059694 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838e067-97ff-4c4c-852e-f462f4ed10de admin admin] PATH_INFO: `/v3/projects/98af09eca26343a3b4b4b540635f8a53/users/28d338cfc46043e4bbce484c7e966428/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:54.065597 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f002767-7e2d-44f7-9c11-cdcccc6861b7 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:54.069725 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce3e9f-715c-4922-b69c-8239b36a14c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:54.070559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce3e9f-715c-4922-b69c-8239b36a14c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:54.070844 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce3e9f-715c-4922-b69c-8239b36a14c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:54.071121 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce3e9f-715c-4922-b69c-8239b36a14c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:54.075012 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a838e067-97ff-4c4c-852e-f462f4ed10de admin admin] RBAC: Authorizing `identity:create_grant(project_id=98af09eca26343a3b4b4b540635f8a53, user_id=28d338cfc46043e4bbce484c7e966428, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:54.077667 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a838e067-97ff-4c4c-852e-f462f4ed10de admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:54.079606 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce3e9f-715c-4922-b69c-8239b36a14c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:54.081110 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce3e9f-715c-4922-b69c-8239b36a14c2 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:54.091011 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1145/2296] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:54 2022] PUT /identity/v3/projects/98af09eca26343a3b4b4b540635f8a53/users/28d338cfc46043e4bbce484c7e966428/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:54.094224 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1152/2297] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:54.103043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838e067-97ff-4c4c-852e-f462f4ed10de admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:54.108250 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce3e9f-715c-4922-b69c-8239b36a14c2 service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:54.119566 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce3e9f-715c-4922-b69c-8239b36a14c2 service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:54.147862 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:54.168008 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838e067-97ff-4c4c-852e-f462f4ed10de admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:54.172621 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bfbdbb-8991-4425-8fc0-432b4d495ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:54.173232 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bfbdbb-8991-4425-8fc0-432b4d495ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:54.173515 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bfbdbb-8991-4425-8fc0-432b4d495ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:54.173793 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bfbdbb-8991-4425-8fc0-432b4d495ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:54.174470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bfbdbb-8991-4425-8fc0-432b4d495ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:54.176074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bfbdbb-8991-4425-8fc0-432b4d495ea7 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:54.183787 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1f808-b5e0-46cb-af6f-fcf3c9caadf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:54.184426 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1f808-b5e0-46cb-af6f-fcf3c9caadf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:54.184705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1f808-b5e0-46cb-af6f-fcf3c9caadf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:54.184976 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1f808-b5e0-46cb-af6f-fcf3c9caadf3 admin admin] PATH_INFO: `/v3/users/1a7e3b35d8cb4df88d620c5db7fb426b` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:54.186094 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1f808-b5e0-46cb-af6f-fcf3c9caadf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a7e3b35d8cb4df88d620c5db7fb426b)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:54.186434 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1146/2298] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:54.190509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bfbdbb-8991-4425-8fc0-432b4d495ea7 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:54.191282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1f808-b5e0-46cb-af6f-fcf3c9caadf3 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:54.200526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bfbdbb-8991-4425-8fc0-432b4d495ea7 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:54.205200 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ac892-7a66-4ed5-9c3f-e915e4b2ba9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:54.205948 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ac892-7a66-4ed5-9c3f-e915e4b2ba9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:54.206318 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ac892-7a66-4ed5-9c3f-e915e4b2ba9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:54.206605 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ac892-7a66-4ed5-9c3f-e915e4b2ba9e admin admin] PATH_INFO: `/v3/projects/98af09eca26343a3b4b4b540635f8a53/users/28d338cfc46043e4bbce484c7e966428/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:54.208777 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ac892-7a66-4ed5-9c3f-e915e4b2ba9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=98af09eca26343a3b4b4b540635f8a53, user_id=28d338cfc46043e4bbce484c7e966428, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:54.211955 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ac892-7a66-4ed5-9c3f-e915e4b2ba9e admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:54.222983 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1147/2299] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:54 2022] PUT /identity/v3/projects/98af09eca26343a3b4b4b540635f8a53/users/28d338cfc46043e4bbce484c7e966428/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:54.235435 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ea1e5-2364-48b8-acb9-b533a633dbec None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:54.235870 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ea1e5-2364-48b8-acb9-b533a633dbec None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:54.236147 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ea1e5-2364-48b8-acb9-b533a633dbec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:54.241933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-05d1f808-b5e0-46cb-af6f-fcf3c9caadf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a7e3b35d8cb4df88d620c5db7fb426b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:54.249967 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-05d1f808-b5e0-46cb-af6f-fcf3c9caadf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a7e3b35d8cb4df88d620c5db7fb426b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:54.250680 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-05d1f808-b5e0-46cb-af6f-fcf3c9caadf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a7e3b35d8cb4df88d620c5db7fb426b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:54.263953 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-05d1f808-b5e0-46cb-af6f-fcf3c9caadf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a7e3b35d8cb4df88d620c5db7fb426b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:54.264621 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-05d1f808-b5e0-46cb-af6f-fcf3c9caadf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a7e3b35d8cb4df88d620c5db7fb426b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:54.268448 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-849ea1e5-2364-48b8-acb9-b533a633dbec None None] MFA Rules not processed for user `28d338cfc46043e4bbce484c7e966428`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:54.269804 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-849ea1e5-2364-48b8-acb9-b533a633dbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:54.277886 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1153/2300] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:54 2022] DELETE /identity/v3/users/1a7e3b35d8cb4df88d620c5db7fb426b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:54.282210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457eb36-4235-4bfc-a382-048979788c20 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:54.282619 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457eb36-4235-4bfc-a382-048979788c20 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:54.282996 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457eb36-4235-4bfc-a382-048979788c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:54.302647 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-5457eb36-4235-4bfc-a382-048979788c20 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:54.303764 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-5457eb36-4235-4bfc-a382-048979788c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:54.345050 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1148/2301] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:54 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:35:54.361053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ea1e5-2364-48b8-acb9-b533a633dbec None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:54.362514 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-849ea1e5-2364-48b8-acb9-b533a633dbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:54.379974 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1154/2302] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:35:54.389554 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457eb36-4235-4bfc-a382-048979788c20 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:54.390929 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-5457eb36-4235-4bfc-a382-048979788c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:54.420165 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:54.433276 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ea1e5-2364-48b8-acb9-b533a633dbec None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:54.445050 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:54.450329 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457eb36-4235-4bfc-a382-048979788c20 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:54.454846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951275ff-a2fa-4218-8775-3d4ec1b17fc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:54.455535 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951275ff-a2fa-4218-8775-3d4ec1b17fc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:54.455966 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951275ff-a2fa-4218-8775-3d4ec1b17fc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:54.456462 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951275ff-a2fa-4218-8775-3d4ec1b17fc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:54.467577 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951275ff-a2fa-4218-8775-3d4ec1b17fc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:54.469002 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951275ff-a2fa-4218-8775-3d4ec1b17fc2 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:54.480711 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1155/2303] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:35:54.486568 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b5825-ad69-494f-a942-2e7e268784a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:54.487194 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b5825-ad69-494f-a942-2e7e268784a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:54.487541 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b5825-ad69-494f-a942-2e7e268784a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:54.487746 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b5825-ad69-494f-a942-2e7e268784a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:54.489691 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-540b5825-ad69-494f-a942-2e7e268784a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:54.556098 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b5825-ad69-494f-a942-2e7e268784a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:54.557677 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b5825-ad69-494f-a942-2e7e268784a9 service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:54.571130 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1149/2304] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:35:54.926677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951275ff-a2fa-4218-8775-3d4ec1b17fc2 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:54.928169 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-951275ff-a2fa-4218-8775-3d4ec1b17fc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:54.948469 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951275ff-a2fa-4218-8775-3d4ec1b17fc2 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:54.960397 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8038f-0271-4c0e-b3d4-a3a9e68447d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:54.960397 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8038f-0271-4c0e-b3d4-a3a9e68447d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:54.963403 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8038f-0271-4c0e-b3d4-a3a9e68447d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:54.964011 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8038f-0271-4c0e-b3d4-a3a9e68447d4 admin admin] PATH_INFO: `/v3/projects/a5781da6630a4238b361fcb1754fd237` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:54.966218 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a8038f-0271-4c0e-b3d4-a3a9e68447d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5781da6630a4238b361fcb1754fd237)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:54.968033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a8038f-0271-4c0e-b3d4-a3a9e68447d4 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:55.038171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-48a8038f-0271-4c0e-b3d4-a3a9e68447d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5781da6630a4238b361fcb1754fd237', 'request_id': 'req-48a8038f-0271-4c0e-b3d4-a3a9e68447d4'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:55.046009 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-48a8038f-0271-4c0e-b3d4-a3a9e68447d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5781da6630a4238b361fcb1754fd237', 'request_id': 'req-48a8038f-0271-4c0e-b3d4-a3a9e68447d4'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:55.055812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-48a8038f-0271-4c0e-b3d4-a3a9e68447d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5781da6630a4238b361fcb1754fd237', 'request_id': 'req-48a8038f-0271-4c0e-b3d4-a3a9e68447d4'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:55.067607 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-48a8038f-0271-4c0e-b3d4-a3a9e68447d4 admin admin] The token cache is being invalidate because project a5781da6630a4238b361fcb1754fd237 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:55.067903 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-48a8038f-0271-4c0e-b3d4-a3a9e68447d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:55.069222 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-48a8038f-0271-4c0e-b3d4-a3a9e68447d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:55.071431 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1156/2305] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:54 2022] DELETE /identity/v3/projects/a5781da6630a4238b361fcb1754fd237 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:55.077173 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76acaa59-dbbd-4133-9c5b-aff530ec3b21 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:55.077700 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76acaa59-dbbd-4133-9c5b-aff530ec3b21 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:55.077700 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76acaa59-dbbd-4133-9c5b-aff530ec3b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:55.099916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-76acaa59-dbbd-4133-9c5b-aff530ec3b21 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:55.100988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-76acaa59-dbbd-4133-9c5b-aff530ec3b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:55.166759 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1150/2306] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:35:55.176843 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e714ef4-d734-4dd0-bb85-e977af445eb4 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:55.177142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e714ef4-d734-4dd0-bb85-e977af445eb4 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:55.177404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e714ef4-d734-4dd0-bb85-e977af445eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:55.208170 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-1e714ef4-d734-4dd0-bb85-e977af445eb4 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:55.209604 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-1e714ef4-d734-4dd0-bb85-e977af445eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:55.219508 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1157/2307] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:55.231358 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76acaa59-dbbd-4133-9c5b-aff530ec3b21 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:55.241632 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76acaa59-dbbd-4133-9c5b-aff530ec3b21 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:55.247045 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572629f1-4ec7-42f6-90e4-e7465e525467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:55.247658 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572629f1-4ec7-42f6-90e4-e7465e525467 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:55.247941 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572629f1-4ec7-42f6-90e4-e7465e525467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:55.248219 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572629f1-4ec7-42f6-90e4-e7465e525467 admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:55.248894 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572629f1-4ec7-42f6-90e4-e7465e525467 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:35:55.249200 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572629f1-4ec7-42f6-90e4-e7465e525467 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:55.250847 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572629f1-4ec7-42f6-90e4-e7465e525467 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:55.264028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1151/2308] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:35:55.268021 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e714ef4-d734-4dd0-bb85-e977af445eb4 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:55.277214 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e714ef4-d734-4dd0-bb85-e977af445eb4 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:55.282123 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6599d-8f29-4e59-bea4-26f2135c9543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:55.282762 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6599d-8f29-4e59-bea4-26f2135c9543 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:55.283091 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6599d-8f29-4e59-bea4-26f2135c9543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:55.283396 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6599d-8f29-4e59-bea4-26f2135c9543 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:55.284035 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d6599d-8f29-4e59-bea4-26f2135c9543 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:55.285951 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d6599d-8f29-4e59-bea4-26f2135c9543 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:55.315998 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1158/2309] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:55 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:35:55.320015 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572629f1-4ec7-42f6-90e4-e7465e525467 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:55.330128 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572629f1-4ec7-42f6-90e4-e7465e525467 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:55.372728 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:55.388861 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce9800-246f-4703-84e9-506df1e10229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:55.389473 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce9800-246f-4703-84e9-506df1e10229 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:55.389756 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce9800-246f-4703-84e9-506df1e10229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:55.390025 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce9800-246f-4703-84e9-506df1e10229 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:55.390689 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce9800-246f-4703-84e9-506df1e10229 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:55.392608 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce9800-246f-4703-84e9-506df1e10229 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:55.415734 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1152/2310] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:55 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:35:55.419992 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6599d-8f29-4e59-bea4-26f2135c9543 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:55.432391 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6599d-8f29-4e59-bea4-26f2135c9543 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:55.437132 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644664f-6200-430d-b15e-89a1e3c49b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:55.437744 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644664f-6200-430d-b15e-89a1e3c49b52 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:55.438029 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644664f-6200-430d-b15e-89a1e3c49b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:55.438495 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644664f-6200-430d-b15e-89a1e3c49b52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:55.439265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644664f-6200-430d-b15e-89a1e3c49b52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:55.440775 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644664f-6200-430d-b15e-89a1e3c49b52 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:55.453220 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1159/2311] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:35:55.455075 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce9800-246f-4703-84e9-506df1e10229 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:55.465232 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce9800-246f-4703-84e9-506df1e10229 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:55.470576 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496848e-3a73-40d1-97d9-26262a98e7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:55.471241 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496848e-3a73-40d1-97d9-26262a98e7ee admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:55.471528 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496848e-3a73-40d1-97d9-26262a98e7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:55.471803 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496848e-3a73-40d1-97d9-26262a98e7ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:55.472475 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d496848e-3a73-40d1-97d9-26262a98e7ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:55.474159 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d496848e-3a73-40d1-97d9-26262a98e7ee admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:55.485969 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1153/2312] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:55.490227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644664f-6200-430d-b15e-89a1e3c49b52 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:55.501353 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644664f-6200-430d-b15e-89a1e3c49b52 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:55.506098 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78643906-d7d6-4c42-ae37-5bc792b60d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:55.506811 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78643906-d7d6-4c42-ae37-5bc792b60d34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:55.507096 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78643906-d7d6-4c42-ae37-5bc792b60d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:55.507378 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78643906-d7d6-4c42-ae37-5bc792b60d34 admin admin] PATH_INFO: `/v3/projects/fa7fc5a55f4a41e28aaa945a72088725/users/13193ad3a22244279f9e2e93a2e9ac16/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:55.522990 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78643906-d7d6-4c42-ae37-5bc792b60d34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa7fc5a55f4a41e28aaa945a72088725, user_id=13193ad3a22244279f9e2e93a2e9ac16, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:55.525686 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78643906-d7d6-4c42-ae37-5bc792b60d34 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:55.541852 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1160/2313] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:55 2022] PUT /identity/v3/projects/fa7fc5a55f4a41e28aaa945a72088725/users/13193ad3a22244279f9e2e93a2e9ac16/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:55.552764 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0c5f2-ede7-4994-ab47-5ce48047d8c9 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:55.553052 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0c5f2-ede7-4994-ab47-5ce48047d8c9 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:55.553333 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0c5f2-ede7-4994-ab47-5ce48047d8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:55.581671 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-e9b0c5f2-ede7-4994-ab47-5ce48047d8c9 None None] MFA Rules not processed for user `13193ad3a22244279f9e2e93a2e9ac16`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:55.582866 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-e9b0c5f2-ede7-4994-ab47-5ce48047d8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:55.659246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1154/2314] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:55.672068 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78643906-d7d6-4c42-ae37-5bc792b60d34 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:55.675510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-78643906-d7d6-4c42-ae37-5bc792b60d34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:55.775756 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78643906-d7d6-4c42-ae37-5bc792b60d34 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:55.786007 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2545b-65bb-4c91-8094-f4852e012028 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b09f6d550c9142bf956b225bf5172e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:55.787809 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2545b-65bb-4c91-8094-f4852e012028 service glance] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:55.788480 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2545b-65bb-4c91-8094-f4852e012028 service glance] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:55.788990 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2545b-65bb-4c91-8094-f4852e012028 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:55.805488 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd2545b-65bb-4c91-8094-f4852e012028 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:55.808579 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd2545b-65bb-4c91-8094-f4852e012028 service glance] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:55.828308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1161/2315] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:56.222531 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0c5f2-ede7-4994-ab47-5ce48047d8c9 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:56.223987 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-e9b0c5f2-ede7-4994-ab47-5ce48047d8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:56.289717 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0c5f2-ede7-4994-ab47-5ce48047d8c9 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:56.294105 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26e52d-0048-4a38-ba49-c6cc41c39bf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:56.295455 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26e52d-0048-4a38-ba49-c6cc41c39bf5 service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:56.295713 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26e52d-0048-4a38-ba49-c6cc41c39bf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:56.297467 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26e52d-0048-4a38-ba49-c6cc41c39bf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:56.299317 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-1e26e52d-0048-4a38-ba49-c6cc41c39bf5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:56.386925 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e26e52d-0048-4a38-ba49-c6cc41c39bf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:56.388422 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e26e52d-0048-4a38-ba49-c6cc41c39bf5 service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:56.398718 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1155/2316] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:57.068214 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2545b-65bb-4c91-8094-f4852e012028 service glance] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:57.086626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2545b-65bb-4c91-8094-f4852e012028 service glance] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:57.092462 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb50c5-f886-4a38-b408-1b78b10aad02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:57.093398 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb50c5-f886-4a38-b408-1b78b10aad02 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:57.093757 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb50c5-f886-4a38-b408-1b78b10aad02 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:57.094214 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb50c5-f886-4a38-b408-1b78b10aad02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:57.096228 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-aacb50c5-f886-4a38-b408-1b78b10aad02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:57.109400 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:57.195270 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb50c5-f886-4a38-b408-1b78b10aad02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:57.195648 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb50c5-f886-4a38-b408-1b78b10aad02 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:57.207908 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1162/2317] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:57 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:57.946171 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26e52d-0048-4a38-ba49-c6cc41c39bf5 service cinder] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:57.947738 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-1e26e52d-0048-4a38-ba49-c6cc41c39bf5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:57.954455 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:58.019896 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26e52d-0048-4a38-ba49-c6cc41c39bf5 service cinder] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:58.025044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e4a53-81d3-4283-85a9-740cd7b6a0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:58.025722 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e4a53-81d3-4283-85a9-740cd7b6a0b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:58.026004 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e4a53-81d3-4283-85a9-740cd7b6a0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:58.026304 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e4a53-81d3-4283-85a9-740cd7b6a0b2 admin admin] PATH_INFO: `/v3/users/dbd8aaf124e14138b3aa4257558ff8e6` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:58.027435 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6e4a53-81d3-4283-85a9-740cd7b6a0b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbd8aaf124e14138b3aa4257558ff8e6)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:58.029421 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6e4a53-81d3-4283-85a9-740cd7b6a0b2 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:58.078892 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-0a6e4a53-81d3-4283-85a9-740cd7b6a0b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbd8aaf124e14138b3aa4257558ff8e6'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:58.079537 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-0a6e4a53-81d3-4283-85a9-740cd7b6a0b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbd8aaf124e14138b3aa4257558ff8e6'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:58.087169 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-0a6e4a53-81d3-4283-85a9-740cd7b6a0b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbd8aaf124e14138b3aa4257558ff8e6'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:58.087828 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-0a6e4a53-81d3-4283-85a9-740cd7b6a0b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbd8aaf124e14138b3aa4257558ff8e6'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:58.101114 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-0a6e4a53-81d3-4283-85a9-740cd7b6a0b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbd8aaf124e14138b3aa4257558ff8e6'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:58.115116 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1156/2318] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:57 2022] DELETE /identity/v3/users/dbd8aaf124e14138b3aa4257558ff8e6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:58.119918 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed412fed-fffa-46d3-b981-ec0fc8a9ea89 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:58.120420 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed412fed-fffa-46d3-b981-ec0fc8a9ea89 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:58.120517 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed412fed-fffa-46d3-b981-ec0fc8a9ea89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:58.139574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-ed412fed-fffa-46d3-b981-ec0fc8a9ea89 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:58.140704 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ed412fed-fffa-46d3-b981-ec0fc8a9ea89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:58.208658 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1163/2319] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:35:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:58.218957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e4a53-81d3-4283-85a9-740cd7b6a0b2 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:58.220438 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-0a6e4a53-81d3-4283-85a9-740cd7b6a0b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:58.293159 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e4a53-81d3-4283-85a9-740cd7b6a0b2 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:58.297013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132aa45-5639-4467-a9c9-4f99f0fd3b8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:58.297701 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132aa45-5639-4467-a9c9-4f99f0fd3b8d service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:58.298049 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132aa45-5639-4467-a9c9-4f99f0fd3b8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:58.298417 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132aa45-5639-4467-a9c9-4f99f0fd3b8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:58.308581 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132aa45-5639-4467-a9c9-4f99f0fd3b8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:58.310040 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132aa45-5639-4467-a9c9-4f99f0fd3b8d service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:58.319936 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1157/2320] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:58.635926 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed412fed-fffa-46d3-b981-ec0fc8a9ea89 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:58.640809 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ed412fed-fffa-46d3-b981-ec0fc8a9ea89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:58.660085 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed412fed-fffa-46d3-b981-ec0fc8a9ea89 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:58.666638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba627e-d6c9-489b-b72d-980195610836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:58.676125 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba627e-d6c9-489b-b72d-980195610836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:58.676421 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba627e-d6c9-489b-b72d-980195610836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:58.676837 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba627e-d6c9-489b-b72d-980195610836 admin admin] PATH_INFO: `/v3/projects/10f1f9af43e94b129a12a78888672054` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:58.678568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba627e-d6c9-489b-b72d-980195610836 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10f1f9af43e94b129a12a78888672054)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:58.681037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba627e-d6c9-489b-b72d-980195610836 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:58.746771 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5eba627e-d6c9-489b-b72d-980195610836 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10f1f9af43e94b129a12a78888672054', 'request_id': 'req-5eba627e-d6c9-489b-b72d-980195610836'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:58.754437 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5eba627e-d6c9-489b-b72d-980195610836 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10f1f9af43e94b129a12a78888672054', 'request_id': 'req-5eba627e-d6c9-489b-b72d-980195610836'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:58.764370 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5eba627e-d6c9-489b-b72d-980195610836 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10f1f9af43e94b129a12a78888672054', 'request_id': 'req-5eba627e-d6c9-489b-b72d-980195610836'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:58.775691 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5eba627e-d6c9-489b-b72d-980195610836 admin admin] The token cache is being invalidate because project 10f1f9af43e94b129a12a78888672054 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:58.775986 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5eba627e-d6c9-489b-b72d-980195610836 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:58.776625 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-5eba627e-d6c9-489b-b72d-980195610836 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:58.778273 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1164/2321] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:58 2022] DELETE /identity/v3/projects/10f1f9af43e94b129a12a78888672054 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:58.782320 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132aa45-5639-4467-a9c9-4f99f0fd3b8d service neutron] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:58.792585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132aa45-5639-4467-a9c9-4f99f0fd3b8d service neutron] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:58.825874 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:58.866439 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75d0fb-ddbc-4381-925e-ffbe83f6e813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:58.867693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75d0fb-ddbc-4381-925e-ffbe83f6e813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:58.868009 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75d0fb-ddbc-4381-925e-ffbe83f6e813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:58.868329 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75d0fb-ddbc-4381-925e-ffbe83f6e813 admin admin] PATH_INFO: `/v3/users/adfb7f81e59b429cabfc0f6abb367599` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:58.869828 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75d0fb-ddbc-4381-925e-ffbe83f6e813 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adfb7f81e59b429cabfc0f6abb367599)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:58.871944 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75d0fb-ddbc-4381-925e-ffbe83f6e813 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:58.925427 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5f75d0fb-ddbc-4381-925e-ffbe83f6e813 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adfb7f81e59b429cabfc0f6abb367599'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:58.926180 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5f75d0fb-ddbc-4381-925e-ffbe83f6e813 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adfb7f81e59b429cabfc0f6abb367599'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:58.935148 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5f75d0fb-ddbc-4381-925e-ffbe83f6e813 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adfb7f81e59b429cabfc0f6abb367599'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:58.935954 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5f75d0fb-ddbc-4381-925e-ffbe83f6e813 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adfb7f81e59b429cabfc0f6abb367599'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:58.950541 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5f75d0fb-ddbc-4381-925e-ffbe83f6e813 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adfb7f81e59b429cabfc0f6abb367599'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:58.964400 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1158/2322] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:58 2022] DELETE /identity/v3/users/adfb7f81e59b429cabfc0f6abb367599 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:59.257327 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba627e-d6c9-489b-b72d-980195610836 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:59.259705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-5eba627e-d6c9-489b-b72d-980195610836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:59.273055 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba627e-d6c9-489b-b72d-980195610836 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:59.307632 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:59.337936 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5c810-8799-483c-9edb-bf7b9570735f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:59.338668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5c810-8799-483c-9edb-bf7b9570735f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:59.338954 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5c810-8799-483c-9edb-bf7b9570735f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:59.339226 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5c810-8799-483c-9edb-bf7b9570735f admin admin] PATH_INFO: `/v3/projects/ee27b104875e4322b2c3804495004075` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:59.340349 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a5c810-8799-483c-9edb-bf7b9570735f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee27b104875e4322b2c3804495004075)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:59.342227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a5c810-8799-483c-9edb-bf7b9570735f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:59.391252 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-94a5c810-8799-483c-9edb-bf7b9570735f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee27b104875e4322b2c3804495004075', 'request_id': 'req-94a5c810-8799-483c-9edb-bf7b9570735f'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:59.397714 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-94a5c810-8799-483c-9edb-bf7b9570735f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee27b104875e4322b2c3804495004075', 'request_id': 'req-94a5c810-8799-483c-9edb-bf7b9570735f'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:59.405215 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-94a5c810-8799-483c-9edb-bf7b9570735f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee27b104875e4322b2c3804495004075', 'request_id': 'req-94a5c810-8799-483c-9edb-bf7b9570735f'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:59.418976 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-94a5c810-8799-483c-9edb-bf7b9570735f admin admin] The token cache is being invalidate because project ee27b104875e4322b2c3804495004075 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:59.419279 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-94a5c810-8799-483c-9edb-bf7b9570735f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:59.419953 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-94a5c810-8799-483c-9edb-bf7b9570735f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:59.421420 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1165/2323] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:59 2022] DELETE /identity/v3/projects/ee27b104875e4322b2c3804495004075 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:59.425776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75d0fb-ddbc-4381-925e-ffbe83f6e813 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:59.427489 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-5f75d0fb-ddbc-4381-925e-ffbe83f6e813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:59.442121 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75d0fb-ddbc-4381-925e-ffbe83f6e813 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:59.502104 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c3c00-e04d-4fc7-8c4c-7faef4c9e262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:59.502770 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c3c00-e04d-4fc7-8c4c-7faef4c9e262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:59.503050 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c3c00-e04d-4fc7-8c4c-7faef4c9e262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:59.503324 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c3c00-e04d-4fc7-8c4c-7faef4c9e262 admin admin] PATH_INFO: `/v3/users/81074f50771f46219dfde939015a5699` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:59.504467 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c3c00-e04d-4fc7-8c4c-7faef4c9e262 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81074f50771f46219dfde939015a5699)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:59.506412 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c3c00-e04d-4fc7-8c4c-7faef4c9e262 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:59.558550 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-7e8c3c00-e04d-4fc7-8c4c-7faef4c9e262 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81074f50771f46219dfde939015a5699'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:59.559651 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-7e8c3c00-e04d-4fc7-8c4c-7faef4c9e262 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81074f50771f46219dfde939015a5699'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:59.565053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:59.571714 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-7e8c3c00-e04d-4fc7-8c4c-7faef4c9e262 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81074f50771f46219dfde939015a5699'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:59.572575 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-7e8c3c00-e04d-4fc7-8c4c-7faef4c9e262 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81074f50771f46219dfde939015a5699'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:59.585360 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-7e8c3c00-e04d-4fc7-8c4c-7faef4c9e262 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81074f50771f46219dfde939015a5699'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:59.597155 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1159/2324] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:59 2022] DELETE /identity/v3/users/81074f50771f46219dfde939015a5699 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:59.889912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5c810-8799-483c-9edb-bf7b9570735f admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:59.891430 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-94a5c810-8799-483c-9edb-bf7b9570735f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:59.904709 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5c810-8799-483c-9edb-bf7b9570735f admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:59.962873 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d1c6d-a3f8-48e5-b8ce-43a0c934fcd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:59.963428 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d1c6d-a3f8-48e5-b8ce-43a0c934fcd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:59.963712 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d1c6d-a3f8-48e5-b8ce-43a0c934fcd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:59.963987 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d1c6d-a3f8-48e5-b8ce-43a0c934fcd7 admin admin] PATH_INFO: `/v3/projects/9db4031bf080409da863959a894954ca` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:59.965115 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d1c6d-a3f8-48e5-b8ce-43a0c934fcd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9db4031bf080409da863959a894954ca)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:59.967042 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d1c6d-a3f8-48e5-b8ce-43a0c934fcd7 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:00.015900 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7f7d1c6d-a3f8-48e5-b8ce-43a0c934fcd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9db4031bf080409da863959a894954ca', 'request_id': 'req-7f7d1c6d-a3f8-48e5-b8ce-43a0c934fcd7'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:00.025944 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7f7d1c6d-a3f8-48e5-b8ce-43a0c934fcd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9db4031bf080409da863959a894954ca', 'request_id': 'req-7f7d1c6d-a3f8-48e5-b8ce-43a0c934fcd7'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:00.037825 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7f7d1c6d-a3f8-48e5-b8ce-43a0c934fcd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9db4031bf080409da863959a894954ca', 'request_id': 'req-7f7d1c6d-a3f8-48e5-b8ce-43a0c934fcd7'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:00.040526 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:00.053870 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7f7d1c6d-a3f8-48e5-b8ce-43a0c934fcd7 admin admin] The token cache is being invalidate because project 9db4031bf080409da863959a894954ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:36:00.053870 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7f7d1c6d-a3f8-48e5-b8ce-43a0c934fcd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:00.053870 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7f7d1c6d-a3f8-48e5-b8ce-43a0c934fcd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:00.055917 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1166/2325] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:59 2022] DELETE /identity/v3/projects/9db4031bf080409da863959a894954ca => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:00.060654 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05a1d9-dca1-4012-ab0a-a1e21fcfce06 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:00.061129 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05a1d9-dca1-4012-ab0a-a1e21fcfce06 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:00.061387 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05a1d9-dca1-4012-ab0a-a1e21fcfce06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:00.082755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-7c05a1d9-dca1-4012-ab0a-a1e21fcfce06 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:00.083999 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-7c05a1d9-dca1-4012-ab0a-a1e21fcfce06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:00.150559 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1160/2326] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:00.160058 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bfd54-9d42-443d-89d3-e7218e7ccb8c None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:00.160351 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bfd54-9d42-443d-89d3-e7218e7ccb8c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:00.160629 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bfd54-9d42-443d-89d3-e7218e7ccb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:00.183260 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-fa3bfd54-9d42-443d-89d3-e7218e7ccb8c None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:00.184577 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-fa3bfd54-9d42-443d-89d3-e7218e7ccb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:00.194140 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1167/2327] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:00.198123 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05a1d9-dca1-4012-ab0a-a1e21fcfce06 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:00.209293 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05a1d9-dca1-4012-ab0a-a1e21fcfce06 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:00.220379 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc771997-7eed-415b-85d2-8132ac2b9dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:00.220379 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc771997-7eed-415b-85d2-8132ac2b9dcc admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:00.220379 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc771997-7eed-415b-85d2-8132ac2b9dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:00.220379 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc771997-7eed-415b-85d2-8132ac2b9dcc admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:00.220961 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc771997-7eed-415b-85d2-8132ac2b9dcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:36:00.220961 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc771997-7eed-415b-85d2-8132ac2b9dcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:00.221583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc771997-7eed-415b-85d2-8132ac2b9dcc admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:00.235352 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1161/2328] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:36:00.239967 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3bfd54-9d42-443d-89d3-e7218e7ccb8c None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:00.249378 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3bfd54-9d42-443d-89d3-e7218e7ccb8c None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:00.254761 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc3d18-506a-424a-a3ad-9980e9140be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:00.255362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc3d18-506a-424a-a3ad-9980e9140be0 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:00.255644 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc3d18-506a-424a-a3ad-9980e9140be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:00.255924 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc3d18-506a-424a-a3ad-9980e9140be0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:00.256561 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dc3d18-506a-424a-a3ad-9980e9140be0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:00.258519 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dc3d18-506a-424a-a3ad-9980e9140be0 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:00.289671 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1168/2329] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:36:00 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:36:00.293970 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc771997-7eed-415b-85d2-8132ac2b9dcc admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:00.305751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc771997-7eed-415b-85d2-8132ac2b9dcc admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:00.371333 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcfe7de-7314-465f-98b1-a8001a7ceac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:00.371972 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfe7de-7314-465f-98b1-a8001a7ceac0 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:00.372251 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfe7de-7314-465f-98b1-a8001a7ceac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:00.372522 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfe7de-7314-465f-98b1-a8001a7ceac0 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:00.373172 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcfe7de-7314-465f-98b1-a8001a7ceac0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:00.375864 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcfe7de-7314-465f-98b1-a8001a7ceac0 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:00.397743 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1162/2330] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:36:00 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:00.402602 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc3d18-506a-424a-a3ad-9980e9140be0 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:00.413839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc3d18-506a-424a-a3ad-9980e9140be0 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:00.420925 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaddff7-89d4-48e6-9783-007df0f327c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:00.421617 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaddff7-89d4-48e6-9783-007df0f327c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:00.421914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaddff7-89d4-48e6-9783-007df0f327c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:00.422260 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaddff7-89d4-48e6-9783-007df0f327c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:00.422961 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaddff7-89d4-48e6-9783-007df0f327c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:00.424792 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaddff7-89d4-48e6-9783-007df0f327c4 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:00.442271 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1169/2331] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:00.445550 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcfe7de-7314-465f-98b1-a8001a7ceac0 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:00.457028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcfe7de-7314-465f-98b1-a8001a7ceac0 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:00.462375 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7fad4b-0d76-4275-bb33-21fd1a37d374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:00.462978 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7fad4b-0d76-4275-bb33-21fd1a37d374 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:00.463264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7fad4b-0d76-4275-bb33-21fd1a37d374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:00.463601 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7fad4b-0d76-4275-bb33-21fd1a37d374 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:00.464229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7fad4b-0d76-4275-bb33-21fd1a37d374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:00.466556 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7fad4b-0d76-4275-bb33-21fd1a37d374 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:00.481512 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1163/2332] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:00.486388 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaddff7-89d4-48e6-9783-007df0f327c4 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:00.497888 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaddff7-89d4-48e6-9783-007df0f327c4 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:00.503114 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48768c28-8889-4aa4-8d7d-15d7efb3ddac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:00.503640 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48768c28-8889-4aa4-8d7d-15d7efb3ddac admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:00.503927 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48768c28-8889-4aa4-8d7d-15d7efb3ddac admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:00.504267 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48768c28-8889-4aa4-8d7d-15d7efb3ddac admin admin] PATH_INFO: `/v3/projects/0f2d27289c1744cb88dc4ee160759421/users/148ceb19943146138c302aa08e9f6834/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:00.519722 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48768c28-8889-4aa4-8d7d-15d7efb3ddac admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f2d27289c1744cb88dc4ee160759421, user_id=148ceb19943146138c302aa08e9f6834, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:00.522439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48768c28-8889-4aa4-8d7d-15d7efb3ddac admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:00.536553 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1170/2333] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:00 2022] PUT /identity/v3/projects/0f2d27289c1744cb88dc4ee160759421/users/148ceb19943146138c302aa08e9f6834/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:36:00.546748 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1e442-5393-4259-abc1-6161fe308078 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:00.547040 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1e442-5393-4259-abc1-6161fe308078 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:00.547325 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1e442-5393-4259-abc1-6161fe308078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:00.576891 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-7ae1e442-5393-4259-abc1-6161fe308078 None None] MFA Rules not processed for user `148ceb19943146138c302aa08e9f6834`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:00.578137 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-7ae1e442-5393-4259-abc1-6161fe308078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:00.598755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:00.656998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1164/2334] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:36:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:00.668830 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48768c28-8889-4aa4-8d7d-15d7efb3ddac admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:00.671031 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-48768c28-8889-4aa4-8d7d-15d7efb3ddac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:00.733735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48768c28-8889-4aa4-8d7d-15d7efb3ddac admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:00.737515 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e0f6d-c340-4de6-aba0-8b9f640f25be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:00.738315 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e0f6d-c340-4de6-aba0-8b9f640f25be service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:00.738607 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e0f6d-c340-4de6-aba0-8b9f640f25be service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:00.738882 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e0f6d-c340-4de6-aba0-8b9f640f25be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:00.747346 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e0f6d-c340-4de6-aba0-8b9f640f25be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:00.748778 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e0f6d-c340-4de6-aba0-8b9f640f25be service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:00.758136 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1171/2335] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:01.596510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1e442-5393-4259-abc1-6161fe308078 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:01.597950 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-7ae1e442-5393-4259-abc1-6161fe308078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:01.664139 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1e442-5393-4259-abc1-6161fe308078 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:01.668498 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bcb24d-ec66-4856-a484-560ffd292886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:01.669130 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcb24d-ec66-4856-a484-560ffd292886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:01.669410 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcb24d-ec66-4856-a484-560ffd292886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:01.669683 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcb24d-ec66-4856-a484-560ffd292886 admin admin] PATH_INFO: `/v3/users/3a14e6faf6194244810ef5d405a6da8d` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:01.670822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bcb24d-ec66-4856-a484-560ffd292886 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a14e6faf6194244810ef5d405a6da8d)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:01.672745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bcb24d-ec66-4856-a484-560ffd292886 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:01.720817 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c9bcb24d-ec66-4856-a484-560ffd292886 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a14e6faf6194244810ef5d405a6da8d'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:01.721462 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c9bcb24d-ec66-4856-a484-560ffd292886 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a14e6faf6194244810ef5d405a6da8d'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:01.728554 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c9bcb24d-ec66-4856-a484-560ffd292886 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a14e6faf6194244810ef5d405a6da8d'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:01.729160 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c9bcb24d-ec66-4856-a484-560ffd292886 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a14e6faf6194244810ef5d405a6da8d'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:01.741621 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c9bcb24d-ec66-4856-a484-560ffd292886 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a14e6faf6194244810ef5d405a6da8d'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:01.755327 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1165/2336] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:36:01 2022] DELETE /identity/v3/users/3a14e6faf6194244810ef5d405a6da8d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:01.759755 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688dc25-c5ed-409b-b928-0689c036d93e None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:01.760244 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688dc25-c5ed-409b-b928-0689c036d93e None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:01.760438 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688dc25-c5ed-409b-b928-0689c036d93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:01.781487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-e688dc25-c5ed-409b-b928-0689c036d93e None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:01.782738 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-e688dc25-c5ed-409b-b928-0689c036d93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:01.846719 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1172/2337] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:36:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:01.857266 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bcb24d-ec66-4856-a484-560ffd292886 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:01.858796 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-c9bcb24d-ec66-4856-a484-560ffd292886 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:01.910759 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:01.937031 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bcb24d-ec66-4856-a484-560ffd292886 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:01.941285 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa88c1-6653-4a76-ab32-01916b27d373 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:01.942119 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa88c1-6653-4a76-ab32-01916b27d373 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:01.942391 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa88c1-6653-4a76-ab32-01916b27d373 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:01.942742 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa88c1-6653-4a76-ab32-01916b27d373 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:01.954966 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa88c1-6653-4a76-ab32-01916b27d373 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:01.956404 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa88c1-6653-4a76-ab32-01916b27d373 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:01.965813 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1166/2338] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:02.251389 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e688dc25-c5ed-409b-b928-0689c036d93e None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:02.252897 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-e688dc25-c5ed-409b-b928-0689c036d93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:02.267221 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e688dc25-c5ed-409b-b928-0689c036d93e None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:02.272710 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d7774-773f-454f-9e95-de395b0ac961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:02.273370 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d7774-773f-454f-9e95-de395b0ac961 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:02.273651 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d7774-773f-454f-9e95-de395b0ac961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:02.273921 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d7774-773f-454f-9e95-de395b0ac961 admin admin] PATH_INFO: `/v3/projects/d1acfef541384c44bd383879051f20d0` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:02.275108 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d7774-773f-454f-9e95-de395b0ac961 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1acfef541384c44bd383879051f20d0)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:02.276974 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d7774-773f-454f-9e95-de395b0ac961 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:02.311984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:02.333534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-718d7774-773f-454f-9e95-de395b0ac961 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1acfef541384c44bd383879051f20d0', 'request_id': 'req-718d7774-773f-454f-9e95-de395b0ac961'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:02.353373 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-718d7774-773f-454f-9e95-de395b0ac961 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1acfef541384c44bd383879051f20d0', 'request_id': 'req-718d7774-773f-454f-9e95-de395b0ac961'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:02.360980 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-718d7774-773f-454f-9e95-de395b0ac961 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1acfef541384c44bd383879051f20d0', 'request_id': 'req-718d7774-773f-454f-9e95-de395b0ac961'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:02.372303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-718d7774-773f-454f-9e95-de395b0ac961 admin admin] The token cache is being invalidate because project d1acfef541384c44bd383879051f20d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:36:02.372597 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-718d7774-773f-454f-9e95-de395b0ac961 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:02.373238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-718d7774-773f-454f-9e95-de395b0ac961 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:02.374785 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1173/2339] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:36:02 2022] DELETE /identity/v3/projects/d1acfef541384c44bd383879051f20d0 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:02.378978 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa88c1-6653-4a76-ab32-01916b27d373 service neutron] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:02.388578 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa88c1-6653-4a76-ab32-01916b27d373 service neutron] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:02.445435 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10a15b-f697-4845-ae69-8864b384cf2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:02.445969 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10a15b-f697-4845-ae69-8864b384cf2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:02.446266 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10a15b-f697-4845-ae69-8864b384cf2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:02.446545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10a15b-f697-4845-ae69-8864b384cf2c admin admin] PATH_INFO: `/v3/users/6816abc4bfdf41d39ef5a93f3fe0ceb0` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:02.447649 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10a15b-f697-4845-ae69-8864b384cf2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6816abc4bfdf41d39ef5a93f3fe0ceb0)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:02.449565 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10a15b-f697-4845-ae69-8864b384cf2c admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:02.495198 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-ad10a15b-f697-4845-ae69-8864b384cf2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6816abc4bfdf41d39ef5a93f3fe0ceb0'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:02.495852 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-ad10a15b-f697-4845-ae69-8864b384cf2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6816abc4bfdf41d39ef5a93f3fe0ceb0'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:02.502909 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-ad10a15b-f697-4845-ae69-8864b384cf2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6816abc4bfdf41d39ef5a93f3fe0ceb0'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:02.503537 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-ad10a15b-f697-4845-ae69-8864b384cf2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6816abc4bfdf41d39ef5a93f3fe0ceb0'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:02.515443 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-ad10a15b-f697-4845-ae69-8864b384cf2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6816abc4bfdf41d39ef5a93f3fe0ceb0'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:02.528168 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1167/2340] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:36:02 2022] DELETE /identity/v3/users/6816abc4bfdf41d39ef5a93f3fe0ceb0 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:02.812854 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d7774-773f-454f-9e95-de395b0ac961 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:02.815550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-718d7774-773f-454f-9e95-de395b0ac961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:02.830128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d7774-773f-454f-9e95-de395b0ac961 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:02.887076 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c41ada-cfe9-4c80-8ba0-cc812206ba34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:02.887711 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c41ada-cfe9-4c80-8ba0-cc812206ba34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:02.887994 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c41ada-cfe9-4c80-8ba0-cc812206ba34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:02.888277 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c41ada-cfe9-4c80-8ba0-cc812206ba34 admin admin] PATH_INFO: `/v3/projects/96d06633b175479d8b0479a72affdcd7` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:02.889389 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c41ada-cfe9-4c80-8ba0-cc812206ba34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96d06633b175479d8b0479a72affdcd7)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:02.891634 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c41ada-cfe9-4c80-8ba0-cc812206ba34 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:02.940145 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-16c41ada-cfe9-4c80-8ba0-cc812206ba34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96d06633b175479d8b0479a72affdcd7', 'request_id': 'req-16c41ada-cfe9-4c80-8ba0-cc812206ba34'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:02.947821 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-16c41ada-cfe9-4c80-8ba0-cc812206ba34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96d06633b175479d8b0479a72affdcd7', 'request_id': 'req-16c41ada-cfe9-4c80-8ba0-cc812206ba34'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:02.955394 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-16c41ada-cfe9-4c80-8ba0-cc812206ba34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96d06633b175479d8b0479a72affdcd7', 'request_id': 'req-16c41ada-cfe9-4c80-8ba0-cc812206ba34'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:02.966289 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-16c41ada-cfe9-4c80-8ba0-cc812206ba34 admin admin] The token cache is being invalidate because project 96d06633b175479d8b0479a72affdcd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:36:02.966596 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-16c41ada-cfe9-4c80-8ba0-cc812206ba34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:02.967229 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-16c41ada-cfe9-4c80-8ba0-cc812206ba34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:02.968977 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1174/2341] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:36:02 2022] DELETE /identity/v3/projects/96d06633b175479d8b0479a72affdcd7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:02.972839 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10a15b-f697-4845-ae69-8864b384cf2c admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:02.974171 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-ad10a15b-f697-4845-ae69-8864b384cf2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:02.987365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10a15b-f697-4845-ae69-8864b384cf2c admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:03.048165 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca74446-4df2-4590-8f91-43bce1845b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:03.048954 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca74446-4df2-4590-8f91-43bce1845b2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:03.049189 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca74446-4df2-4590-8f91-43bce1845b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:03.049465 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca74446-4df2-4590-8f91-43bce1845b2c admin admin] PATH_INFO: `/v3/users/28d338cfc46043e4bbce484c7e966428` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:03.050621 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca74446-4df2-4590-8f91-43bce1845b2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=28d338cfc46043e4bbce484c7e966428)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:03.052544 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca74446-4df2-4590-8f91-43bce1845b2c admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:03.055496 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:03.108115 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-fca74446-4df2-4590-8f91-43bce1845b2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28d338cfc46043e4bbce484c7e966428'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:03.108940 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-fca74446-4df2-4590-8f91-43bce1845b2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28d338cfc46043e4bbce484c7e966428'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:03.120446 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-fca74446-4df2-4590-8f91-43bce1845b2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28d338cfc46043e4bbce484c7e966428'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:03.121119 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-fca74446-4df2-4590-8f91-43bce1845b2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28d338cfc46043e4bbce484c7e966428'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:03.134128 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-fca74446-4df2-4590-8f91-43bce1845b2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28d338cfc46043e4bbce484c7e966428'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:03.146140 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1168/2342] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:36:02 2022] DELETE /identity/v3/users/28d338cfc46043e4bbce484c7e966428 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:03.381700 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c41ada-cfe9-4c80-8ba0-cc812206ba34 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:03.383816 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-16c41ada-cfe9-4c80-8ba0-cc812206ba34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:03.397668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c41ada-cfe9-4c80-8ba0-cc812206ba34 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:03.434575 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca74446-4df2-4590-8f91-43bce1845b2c admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:03.436090 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-fca74446-4df2-4590-8f91-43bce1845b2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:03.452841 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:03.480322 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb7752-dd9d-4584-a9d0-4c18791d08dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:03.480968 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb7752-dd9d-4584-a9d0-4c18791d08dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:03.481321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb7752-dd9d-4584-a9d0-4c18791d08dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:03.481597 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb7752-dd9d-4584-a9d0-4c18791d08dc admin admin] PATH_INFO: `/v3/users/148ceb19943146138c302aa08e9f6834` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:03.482809 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb7752-dd9d-4584-a9d0-4c18791d08dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=148ceb19943146138c302aa08e9f6834)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:03.484858 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb7752-dd9d-4584-a9d0-4c18791d08dc admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:03.521527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca74446-4df2-4590-8f91-43bce1845b2c admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:03.525960 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30b45b-b8bf-4378-aaaa-e4469b490622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:03.526644 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30b45b-b8bf-4378-aaaa-e4469b490622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:03.526924 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30b45b-b8bf-4378-aaaa-e4469b490622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:03.527639 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30b45b-b8bf-4378-aaaa-e4469b490622 admin admin] PATH_INFO: `/v3/projects/98af09eca26343a3b4b4b540635f8a53` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:03.529012 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e30b45b-b8bf-4378-aaaa-e4469b490622 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98af09eca26343a3b4b4b540635f8a53)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:03.531424 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e30b45b-b8bf-4378-aaaa-e4469b490622 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:03.534955 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f7fb7752-dd9d-4584-a9d0-4c18791d08dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '148ceb19943146138c302aa08e9f6834'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:03.544593 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f7fb7752-dd9d-4584-a9d0-4c18791d08dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '148ceb19943146138c302aa08e9f6834'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:03.545661 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f7fb7752-dd9d-4584-a9d0-4c18791d08dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '148ceb19943146138c302aa08e9f6834'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:03.563781 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f7fb7752-dd9d-4584-a9d0-4c18791d08dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '148ceb19943146138c302aa08e9f6834'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:03.565118 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f7fb7752-dd9d-4584-a9d0-4c18791d08dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '148ceb19943146138c302aa08e9f6834'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:03.590277 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1175/2343] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:36:03 2022] DELETE /identity/v3/users/148ceb19943146138c302aa08e9f6834 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:03.595756 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70288f8-d325-402b-81e3-d49fd3c11545 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:03.596220 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70288f8-d325-402b-81e3-d49fd3c11545 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:03.596507 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70288f8-d325-402b-81e3-d49fd3c11545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:03.600536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5e30b45b-b8bf-4378-aaaa-e4469b490622 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98af09eca26343a3b4b4b540635f8a53', 'request_id': 'req-5e30b45b-b8bf-4378-aaaa-e4469b490622'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:03.610466 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5e30b45b-b8bf-4378-aaaa-e4469b490622 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98af09eca26343a3b4b4b540635f8a53', 'request_id': 'req-5e30b45b-b8bf-4378-aaaa-e4469b490622'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:03.619358 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-c70288f8-d325-402b-81e3-d49fd3c11545 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:03.619759 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5e30b45b-b8bf-4378-aaaa-e4469b490622 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98af09eca26343a3b4b4b540635f8a53', 'request_id': 'req-5e30b45b-b8bf-4378-aaaa-e4469b490622'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:03.620579 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c70288f8-d325-402b-81e3-d49fd3c11545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:03.631985 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5e30b45b-b8bf-4378-aaaa-e4469b490622 admin admin] The token cache is being invalidate because project 98af09eca26343a3b4b4b540635f8a53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:36:03.632304 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5e30b45b-b8bf-4378-aaaa-e4469b490622 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:03.633008 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5e30b45b-b8bf-4378-aaaa-e4469b490622 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:03.635034 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1169/2344] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:36:03 2022] DELETE /identity/v3/projects/98af09eca26343a3b4b4b540635f8a53 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:36:03.639305 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1bf3e-3195-403f-9ccc-82cac2f5257b None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:03.639589 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1bf3e-3195-403f-9ccc-82cac2f5257b None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:03.639871 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1bf3e-3195-403f-9ccc-82cac2f5257b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:03.659371 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-84b1bf3e-3195-403f-9ccc-82cac2f5257b None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:03.660592 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-84b1bf3e-3195-403f-9ccc-82cac2f5257b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:03.666673 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:03.688853 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1176/2345] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:36:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:03.698706 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70288f8-d325-402b-81e3-d49fd3c11545 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:03.700115 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c70288f8-d325-402b-81e3-d49fd3c11545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:03.730122 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1170/2346] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:36:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:36:03.738484 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400be378-6272-460a-b9c9-49f80fad6d47 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:03.738774 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400be378-6272-460a-b9c9-49f80fad6d47 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:03.739068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400be378-6272-460a-b9c9-49f80fad6d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:03.761792 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70288f8-d325-402b-81e3-d49fd3c11545 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:03.762440 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-400be378-6272-460a-b9c9-49f80fad6d47 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:03.763678 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-400be378-6272-460a-b9c9-49f80fad6d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:03.765456 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f155ed7-9618-4b5e-87c3-4b562b3bbaf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:03.766070 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f155ed7-9618-4b5e-87c3-4b562b3bbaf7 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:03.766978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f155ed7-9618-4b5e-87c3-4b562b3bbaf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:03.767262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f155ed7-9618-4b5e-87c3-4b562b3bbaf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:03.772223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1171/2347] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:36:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:36:03.776476 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400be378-6272-460a-b9c9-49f80fad6d47 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:03.776895 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f155ed7-9618-4b5e-87c3-4b562b3bbaf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:03.778368 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f155ed7-9618-4b5e-87c3-4b562b3bbaf7 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:03.792370 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400be378-6272-460a-b9c9-49f80fad6d47 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:03.792760 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1177/2348] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:36:03.798815 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61019d80-8034-4d07-9146-9bd51ef6699d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:03.799435 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61019d80-8034-4d07-9146-9bd51ef6699d admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:03.799742 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61019d80-8034-4d07-9146-9bd51ef6699d admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:03.800014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61019d80-8034-4d07-9146-9bd51ef6699d admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:03.800668 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61019d80-8034-4d07-9146-9bd51ef6699d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:36:03.800967 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61019d80-8034-4d07-9146-9bd51ef6699d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:03.802603 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61019d80-8034-4d07-9146-9bd51ef6699d admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:03.815213 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1172/2349] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:36:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:36:03.819667 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f155ed7-9618-4b5e-87c3-4b562b3bbaf7 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:03.829460 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f155ed7-9618-4b5e-87c3-4b562b3bbaf7 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:03.834179 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6e157-85d1-4b8e-9b31-e0c46dc21c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:03.834814 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6e157-85d1-4b8e-9b31-e0c46dc21c4b admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:03.835095 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6e157-85d1-4b8e-9b31-e0c46dc21c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:03.835369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6e157-85d1-4b8e-9b31-e0c46dc21c4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:03.836645 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b6e157-85d1-4b8e-9b31-e0c46dc21c4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:03.838583 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b6e157-85d1-4b8e-9b31-e0c46dc21c4b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:03.843556 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:03.871422 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1178/2350] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:36:03 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:36:03.875907 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61019d80-8034-4d07-9146-9bd51ef6699d admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:03.888078 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61019d80-8034-4d07-9146-9bd51ef6699d admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:03.966185 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4882d4-2cfa-4364-87d4-5734bd71fd11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:03.966899 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4882d4-2cfa-4364-87d4-5734bd71fd11 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:03.967181 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4882d4-2cfa-4364-87d4-5734bd71fd11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:03.967456 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4882d4-2cfa-4364-87d4-5734bd71fd11 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:03.968132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4882d4-2cfa-4364-87d4-5734bd71fd11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:03.970177 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4882d4-2cfa-4364-87d4-5734bd71fd11 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:03.995240 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1173/2351] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:36:03 2022] POST /identity/v3/users => generated 291 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:03.996907 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6e157-85d1-4b8e-9b31-e0c46dc21c4b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:04.007899 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6e157-85d1-4b8e-9b31-e0c46dc21c4b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:04.012889 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c8c87-aebd-4926-8c78-2d7be62bed3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:04.013510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c8c87-aebd-4926-8c78-2d7be62bed3c admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:04.013792 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c8c87-aebd-4926-8c78-2d7be62bed3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:04.014072 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c8c87-aebd-4926-8c78-2d7be62bed3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:04.014748 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2c8c87-aebd-4926-8c78-2d7be62bed3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:04.016335 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2c8c87-aebd-4926-8c78-2d7be62bed3c admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:04.026957 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1179/2352] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:04.031196 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4882d4-2cfa-4364-87d4-5734bd71fd11 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:04.044285 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4882d4-2cfa-4364-87d4-5734bd71fd11 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:04.055822 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4051bff-6c8b-4742-a97c-a7685f4184f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:04.056803 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4051bff-6c8b-4742-a97c-a7685f4184f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:04.057223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4051bff-6c8b-4742-a97c-a7685f4184f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:04.057573 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4051bff-6c8b-4742-a97c-a7685f4184f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:04.058555 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4051bff-6c8b-4742-a97c-a7685f4184f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:04.060442 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4051bff-6c8b-4742-a97c-a7685f4184f1 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:04.074699 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c8c87-aebd-4926-8c78-2d7be62bed3c admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:04.076488 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1174/2353] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:04.076634 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0d2c8c87-aebd-4926-8c78-2d7be62bed3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:04.081267 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4051bff-6c8b-4742-a97c-a7685f4184f1 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:04.091746 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c8c87-aebd-4926-8c78-2d7be62bed3c admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:04.094180 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4051bff-6c8b-4742-a97c-a7685f4184f1 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:04.097397 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5decac7-22cd-4a66-a7df-dd198b8985cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:04.098035 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5decac7-22cd-4a66-a7df-dd198b8985cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:04.098355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5decac7-22cd-4a66-a7df-dd198b8985cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:04.098632 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5decac7-22cd-4a66-a7df-dd198b8985cb admin admin] PATH_INFO: `/v3/projects/0f2d27289c1744cb88dc4ee160759421` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:04.099493 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89322b59-b5bc-4c84-8d7b-3ef0f21f1547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:04.099853 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5decac7-22cd-4a66-a7df-dd198b8985cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f2d27289c1744cb88dc4ee160759421)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:04.100330 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89322b59-b5bc-4c84-8d7b-3ef0f21f1547 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:04.100842 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89322b59-b5bc-4c84-8d7b-3ef0f21f1547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:04.101777 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89322b59-b5bc-4c84-8d7b-3ef0f21f1547 admin admin] PATH_INFO: `/v3/projects/09b1f2c2ff4e4338acd2a3367a4fdee5/users/b4d4d3b4817e499196e01d1df96b4775/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:04.102359 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5decac7-22cd-4a66-a7df-dd198b8985cb admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:04.120956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89322b59-b5bc-4c84-8d7b-3ef0f21f1547 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09b1f2c2ff4e4338acd2a3367a4fdee5, user_id=b4d4d3b4817e499196e01d1df96b4775, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:04.123786 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89322b59-b5bc-4c84-8d7b-3ef0f21f1547 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:04.149720 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1175/2354] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:04 2022] PUT /identity/v3/projects/09b1f2c2ff4e4338acd2a3367a4fdee5/users/b4d4d3b4817e499196e01d1df96b4775/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:04.154954 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2ff83-5c1e-4f88-8055-c690d72e5dff None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:04.155244 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2ff83-5c1e-4f88-8055-c690d72e5dff None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:04.155652 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2ff83-5c1e-4f88-8055-c690d72e5dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:04.169230 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d5decac7-22cd-4a66-a7df-dd198b8985cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f2d27289c1744cb88dc4ee160759421', 'request_id': 'req-d5decac7-22cd-4a66-a7df-dd198b8985cb'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:04.178542 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d5decac7-22cd-4a66-a7df-dd198b8985cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f2d27289c1744cb88dc4ee160759421', 'request_id': 'req-d5decac7-22cd-4a66-a7df-dd198b8985cb'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:04.179550 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-32d2ff83-5c1e-4f88-8055-c690d72e5dff None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:04.180931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-32d2ff83-5c1e-4f88-8055-c690d72e5dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:04.189167 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d5decac7-22cd-4a66-a7df-dd198b8985cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f2d27289c1744cb88dc4ee160759421', 'request_id': 'req-d5decac7-22cd-4a66-a7df-dd198b8985cb'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:04.204203 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d5decac7-22cd-4a66-a7df-dd198b8985cb admin admin] The token cache is being invalidate because project 0f2d27289c1744cb88dc4ee160759421 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:36:04.204590 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d5decac7-22cd-4a66-a7df-dd198b8985cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:04.205355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-d5decac7-22cd-4a66-a7df-dd198b8985cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:04.208003 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1180/2355] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:36:04 2022] DELETE /identity/v3/projects/0f2d27289c1744cb88dc4ee160759421 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:04.213572 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2f562-97f0-40d1-bef0-5985a4813666 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:04.213885 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2f562-97f0-40d1-bef0-5985a4813666 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:04.214362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2f562-97f0-40d1-bef0-5985a4813666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:04.250627 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-92b2f562-97f0-40d1-bef0-5985a4813666 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:04.251877 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-92b2f562-97f0-40d1-bef0-5985a4813666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:04.255657 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:04.276155 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1176/2356] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:36:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:04.287804 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2ff83-5c1e-4f88-8055-c690d72e5dff None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:04.289488 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-32d2ff83-5c1e-4f88-8055-c690d72e5dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:04.323191 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1181/2357] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:36:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:36:04.331529 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ea434-7981-41d6-9a09-fcda321bf182 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:04.331814 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ea434-7981-41d6-9a09-fcda321bf182 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:04.332084 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ea434-7981-41d6-9a09-fcda321bf182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:04.352328 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-ec5ea434-7981-41d6-9a09-fcda321bf182 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:04.353794 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ec5ea434-7981-41d6-9a09-fcda321bf182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:04.356897 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2ff83-5c1e-4f88-8055-c690d72e5dff None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:04.361188 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030518e6-fc23-4bc9-a4a3-946dbba926be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:04.361820 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030518e6-fc23-4bc9-a4a3-946dbba926be service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:04.362097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030518e6-fc23-4bc9-a4a3-946dbba926be service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:04.365610 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030518e6-fc23-4bc9-a4a3-946dbba926be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:04.365850 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1182/2358] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:36:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:36:04.366535 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ea434-7981-41d6-9a09-fcda321bf182 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:04.372263 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030518e6-fc23-4bc9-a4a3-946dbba926be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:04.373941 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030518e6-fc23-4bc9-a4a3-946dbba926be service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:04.376270 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ea434-7981-41d6-9a09-fcda321bf182 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:04.381553 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce540f4e-588f-46bb-88d1-5828d18d30f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:04.382787 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce540f4e-588f-46bb-88d1-5828d18d30f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:04.383100 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce540f4e-588f-46bb-88d1-5828d18d30f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:04.383374 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce540f4e-588f-46bb-88d1-5828d18d30f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:04.384072 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce540f4e-588f-46bb-88d1-5828d18d30f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:36:04.384369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce540f4e-588f-46bb-88d1-5828d18d30f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:04.386100 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce540f4e-588f-46bb-88d1-5828d18d30f3 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:04.389067 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1177/2359] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:36:04.398835 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1183/2360] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:36:04.403201 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030518e6-fc23-4bc9-a4a3-946dbba926be service neutron] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:04.413212 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030518e6-fc23-4bc9-a4a3-946dbba926be service neutron] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:04.419316 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3ebc0-fe9d-4dee-a309-42bdc6488933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:04.419923 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3ebc0-fe9d-4dee-a309-42bdc6488933 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:04.420199 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3ebc0-fe9d-4dee-a309-42bdc6488933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:04.420465 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3ebc0-fe9d-4dee-a309-42bdc6488933 admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:04.421299 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e3ebc0-fe9d-4dee-a309-42bdc6488933 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:04.423299 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e3ebc0-fe9d-4dee-a309-42bdc6488933 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:04.456510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1178/2361] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:36:04 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:36:04.458292 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce540f4e-588f-46bb-88d1-5828d18d30f3 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:04.471739 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce540f4e-588f-46bb-88d1-5828d18d30f3 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:04.508520 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:04.537460 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a71f28-75e4-4331-b133-164c161e7bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:04.538100 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a71f28-75e4-4331-b133-164c161e7bd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:04.538455 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a71f28-75e4-4331-b133-164c161e7bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:04.539321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a71f28-75e4-4331-b133-164c161e7bd2 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:04.539992 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a71f28-75e4-4331-b133-164c161e7bd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:04.542108 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a71f28-75e4-4331-b133-164c161e7bd2 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:04.567640 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1184/2362] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:36:04 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:36:04.570061 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3ebc0-fe9d-4dee-a309-42bdc6488933 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:04.581569 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3ebc0-fe9d-4dee-a309-42bdc6488933 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:04.587836 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6752b-b21d-4aa5-8e40-0f7e38cad7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:04.588656 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6752b-b21d-4aa5-8e40-0f7e38cad7a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:04.590650 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6752b-b21d-4aa5-8e40-0f7e38cad7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:04.591133 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6752b-b21d-4aa5-8e40-0f7e38cad7a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:04.591929 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe6752b-b21d-4aa5-8e40-0f7e38cad7a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:04.593895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe6752b-b21d-4aa5-8e40-0f7e38cad7a5 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:04.614980 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1179/2363] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:36:04.619108 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a71f28-75e4-4331-b133-164c161e7bd2 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:04.629016 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a71f28-75e4-4331-b133-164c161e7bd2 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:04.634012 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f27bf4b-a8d1-4e91-8ce3-eba31d735d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:04.634662 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27bf4b-a8d1-4e91-8ce3-eba31d735d9f admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:04.634947 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27bf4b-a8d1-4e91-8ce3-eba31d735d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:04.635227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27bf4b-a8d1-4e91-8ce3-eba31d735d9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:04.635874 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f27bf4b-a8d1-4e91-8ce3-eba31d735d9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:04.637765 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f27bf4b-a8d1-4e91-8ce3-eba31d735d9f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:04.652723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1185/2364] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:04.654192 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6752b-b21d-4aa5-8e40-0f7e38cad7a5 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:04.665886 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6752b-b21d-4aa5-8e40-0f7e38cad7a5 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:04.671109 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36780ba-7322-413f-a39e-cba3880658f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:04.672022 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36780ba-7322-413f-a39e-cba3880658f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:04.672208 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36780ba-7322-413f-a39e-cba3880658f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:04.672577 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36780ba-7322-413f-a39e-cba3880658f6 admin admin] PATH_INFO: `/v3/projects/03a9ae2236424b4983a8fb74740d100c/users/5b494d44aec049b79150363147dbb3cb/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:04.700551 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36780ba-7322-413f-a39e-cba3880658f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03a9ae2236424b4983a8fb74740d100c, user_id=5b494d44aec049b79150363147dbb3cb, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:04.703338 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36780ba-7322-413f-a39e-cba3880658f6 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:04.719382 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1180/2365] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:04 2022] PUT /identity/v3/projects/03a9ae2236424b4983a8fb74740d100c/users/5b494d44aec049b79150363147dbb3cb/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:04.732045 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4565d6-34b5-4e71-a1e1-4dbd852f64f5 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:04.732338 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4565d6-34b5-4e71-a1e1-4dbd852f64f5 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:04.732614 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4565d6-34b5-4e71-a1e1-4dbd852f64f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:04.771034 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-1c4565d6-34b5-4e71-a1e1-4dbd852f64f5 None None] MFA Rules not processed for user `5b494d44aec049b79150363147dbb3cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:04.772417 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-1c4565d6-34b5-4e71-a1e1-4dbd852f64f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:04.847277 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1186/2366] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:36:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:04.860021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36780ba-7322-413f-a39e-cba3880658f6 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:04.861783 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-f36780ba-7322-413f-a39e-cba3880658f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:04.933292 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36780ba-7322-413f-a39e-cba3880658f6 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:04.937147 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9196f059-ba51-4335-86e1-e289268d2165 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:04.937832 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196f059-ba51-4335-86e1-e289268d2165 service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:04.938220 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196f059-ba51-4335-86e1-e289268d2165 service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:04.938581 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196f059-ba51-4335-86e1-e289268d2165 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:04.949142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9196f059-ba51-4335-86e1-e289268d2165 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:04.950627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9196f059-ba51-4335-86e1-e289268d2165 service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:04.960239 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1181/2367] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:04.985979 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4565d6-34b5-4e71-a1e1-4dbd852f64f5 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:05.053080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4565d6-34b5-4e71-a1e1-4dbd852f64f5 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:05.067271 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b87f57b-44c0-4ba8-8032-c20a744461df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:05.068618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87f57b-44c0-4ba8-8032-c20a744461df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:05.068618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87f57b-44c0-4ba8-8032-c20a744461df admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:05.070799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87f57b-44c0-4ba8-8032-c20a744461df admin admin] PATH_INFO: `/v3/users/5b494d44aec049b79150363147dbb3cb` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:05.071059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b87f57b-44c0-4ba8-8032-c20a744461df admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b494d44aec049b79150363147dbb3cb)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:05.078271 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b87f57b-44c0-4ba8-8032-c20a744461df admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:05.104901 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9cf6a2-1569-4cc1-ac61-6542347a818b None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:05.105469 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9cf6a2-1569-4cc1-ac61-6542347a818b None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:05.105469 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9cf6a2-1569-4cc1-ac61-6542347a818b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:05.136915 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-bd9cf6a2-1569-4cc1-ac61-6542347a818b None None] MFA Rules not processed for user `b4d4d3b4817e499196e01d1df96b4775`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:05.138556 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-bd9cf6a2-1569-4cc1-ac61-6542347a818b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:05.142006 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7b87f57b-44c0-4ba8-8032-c20a744461df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b494d44aec049b79150363147dbb3cb'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:05.143889 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:05.153890 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7b87f57b-44c0-4ba8-8032-c20a744461df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b494d44aec049b79150363147dbb3cb'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:05.155066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7b87f57b-44c0-4ba8-8032-c20a744461df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b494d44aec049b79150363147dbb3cb'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:05.173084 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7b87f57b-44c0-4ba8-8032-c20a744461df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b494d44aec049b79150363147dbb3cb'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:05.174609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:05.175221 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7b87f57b-44c0-4ba8-8032-c20a744461df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b494d44aec049b79150363147dbb3cb'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:05.196591 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1187/2368] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:36:04 2022] DELETE /identity/v3/users/5b494d44aec049b79150363147dbb3cb => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:05.201543 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c676a02-9770-43ea-884c-d601a80f4305 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:05.201915 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c676a02-9770-43ea-884c-d601a80f4305 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:05.202318 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c676a02-9770-43ea-884c-d601a80f4305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:05.227411 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-0c676a02-9770-43ea-884c-d601a80f4305 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:05.229031 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0c676a02-9770-43ea-884c-d601a80f4305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:05.233123 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1182/2369] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:36:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:36:05.290900 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9cf6a2-1569-4cc1-ac61-6542347a818b None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:05.292703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-bd9cf6a2-1569-4cc1-ac61-6542347a818b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:05.307779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9cf6a2-1569-4cc1-ac61-6542347a818b None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:05.319229 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1188/2370] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:36:05.328981 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c676a02-9770-43ea-884c-d601a80f4305 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:05.330489 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0c676a02-9770-43ea-884c-d601a80f4305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:05.399922 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ec22c-7286-4ea1-b0b1-ab883997799b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:05.401189 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ec22c-7286-4ea1-b0b1-ab883997799b service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:05.401666 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ec22c-7286-4ea1-b0b1-ab883997799b service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:05.402358 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ec22c-7286-4ea1-b0b1-ab883997799b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:05.403261 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c676a02-9770-43ea-884c-d601a80f4305 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:05.408708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27a3f08-68e6-4ff2-ae72-b580243484c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:05.410267 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27a3f08-68e6-4ff2-ae72-b580243484c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:05.410267 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27a3f08-68e6-4ff2-ae72-b580243484c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:05.410267 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27a3f08-68e6-4ff2-ae72-b580243484c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:05.420515 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27a3f08-68e6-4ff2-ae72-b580243484c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:05.422060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27a3f08-68e6-4ff2-ae72-b580243484c9 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:05.432647 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1189/2371] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:36:05.497032 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ec22c-7286-4ea1-b0b1-ab883997799b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:05.498784 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ec22c-7286-4ea1-b0b1-ab883997799b service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:05.509826 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1183/2372] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:05 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:05.765462 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27a3f08-68e6-4ff2-ae72-b580243484c9 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:05.768420 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a27a3f08-68e6-4ff2-ae72-b580243484c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:05.807802 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27a3f08-68e6-4ff2-ae72-b580243484c9 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:05.822834 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03abf2a3-64e9-4dd2-8e63-6fa20a352818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:05.823611 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03abf2a3-64e9-4dd2-8e63-6fa20a352818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:05.823964 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03abf2a3-64e9-4dd2-8e63-6fa20a352818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:05.824316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03abf2a3-64e9-4dd2-8e63-6fa20a352818 admin admin] PATH_INFO: `/v3/projects/03a9ae2236424b4983a8fb74740d100c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:05.825619 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03abf2a3-64e9-4dd2-8e63-6fa20a352818 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03a9ae2236424b4983a8fb74740d100c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:05.827645 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03abf2a3-64e9-4dd2-8e63-6fa20a352818 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:05.886742 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-03abf2a3-64e9-4dd2-8e63-6fa20a352818 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03a9ae2236424b4983a8fb74740d100c', 'request_id': 'req-03abf2a3-64e9-4dd2-8e63-6fa20a352818'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:05.893693 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-03abf2a3-64e9-4dd2-8e63-6fa20a352818 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03a9ae2236424b4983a8fb74740d100c', 'request_id': 'req-03abf2a3-64e9-4dd2-8e63-6fa20a352818'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:05.901611 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-03abf2a3-64e9-4dd2-8e63-6fa20a352818 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03a9ae2236424b4983a8fb74740d100c', 'request_id': 'req-03abf2a3-64e9-4dd2-8e63-6fa20a352818'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:05.903886 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:INSERT=2,keystone:DELETE=9,keystone:SELECT=89 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:05.916841 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-03abf2a3-64e9-4dd2-8e63-6fa20a352818 admin admin] The token cache is being invalidate because project 03a9ae2236424b4983a8fb74740d100c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:36:05.917299 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-03abf2a3-64e9-4dd2-8e63-6fa20a352818 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:05.918184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-03abf2a3-64e9-4dd2-8e63-6fa20a352818 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:05.919908 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1190/2373] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:36:05 2022] DELETE /identity/v3/projects/03a9ae2236424b4983a8fb74740d100c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:05.926581 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516603e-335b-4c60-8113-2a02cd9e9267 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:05.927027 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516603e-335b-4c60-8113-2a02cd9e9267 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:05.927390 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516603e-335b-4c60-8113-2a02cd9e9267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:05.952365 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-b516603e-335b-4c60-8113-2a02cd9e9267 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:05.953724 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-b516603e-335b-4c60-8113-2a02cd9e9267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:06.020147 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1184/2374] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:06.028311 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6da48a-c7e7-46fb-9765-195b3156ad8d None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.028600 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6da48a-c7e7-46fb-9765-195b3156ad8d None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.028878 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6da48a-c7e7-46fb-9765-195b3156ad8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.057291 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-3e6da48a-c7e7-46fb-9765-195b3156ad8d None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:06.058837 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3e6da48a-c7e7-46fb-9765-195b3156ad8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:06.069791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1191/2375] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:36:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:06.074522 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516603e-335b-4c60-8113-2a02cd9e9267 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:06.084917 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516603e-335b-4c60-8113-2a02cd9e9267 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:06.090792 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c096500-2a55-41d5-8998-33917c095150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:06.091340 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c096500-2a55-41d5-8998-33917c095150 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.091619 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c096500-2a55-41d5-8998-33917c095150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.091968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c096500-2a55-41d5-8998-33917c095150 admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.092547 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c096500-2a55-41d5-8998-33917c095150 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:36:06.092852 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c096500-2a55-41d5-8998-33917c095150 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:06.094486 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c096500-2a55-41d5-8998-33917c095150 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:06.108615 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1185/2376] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:36:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:36:06.113792 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6da48a-c7e7-46fb-9765-195b3156ad8d None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:06.128619 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6da48a-c7e7-46fb-9765-195b3156ad8d None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:06.135814 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60765c9b-7b65-410a-a061-c7d347d5af82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:06.136635 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60765c9b-7b65-410a-a061-c7d347d5af82 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.136920 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60765c9b-7b65-410a-a061-c7d347d5af82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.137341 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60765c9b-7b65-410a-a061-c7d347d5af82 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.138187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60765c9b-7b65-410a-a061-c7d347d5af82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:06.140332 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60765c9b-7b65-410a-a061-c7d347d5af82 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:06.174066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1192/2377] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:36:06 2022] POST /identity/v3/projects => generated 379 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:06.178706 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c096500-2a55-41d5-8998-33917c095150 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:06.189300 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c096500-2a55-41d5-8998-33917c095150 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:06.193328 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:06.251781 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c52942-b465-44a1-8985-a9391abe5c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:06.252385 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c52942-b465-44a1-8985-a9391abe5c97 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.252666 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c52942-b465-44a1-8985-a9391abe5c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.252936 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c52942-b465-44a1-8985-a9391abe5c97 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.253560 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c52942-b465-44a1-8985-a9391abe5c97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:06.255519 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c52942-b465-44a1-8985-a9391abe5c97 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:06.276164 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1186/2378] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:36:06 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:06.280299 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60765c9b-7b65-410a-a061-c7d347d5af82 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:06.291820 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60765c9b-7b65-410a-a061-c7d347d5af82 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:06.296841 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc332f6-8206-464a-bd28-f6050270c3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:06.297549 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc332f6-8206-464a-bd28-f6050270c3fb admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.297841 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc332f6-8206-464a-bd28-f6050270c3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.298160 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc332f6-8206-464a-bd28-f6050270c3fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.298908 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc332f6-8206-464a-bd28-f6050270c3fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:06.301228 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc332f6-8206-464a-bd28-f6050270c3fb admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:06.311968 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1193/2379] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:06.316004 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c52942-b465-44a1-8985-a9391abe5c97 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:06.326511 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c52942-b465-44a1-8985-a9391abe5c97 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:06.331482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce7f23-89f2-4150-b18a-19306c0af8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:06.332092 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce7f23-89f2-4150-b18a-19306c0af8d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.332380 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce7f23-89f2-4150-b18a-19306c0af8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.332656 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce7f23-89f2-4150-b18a-19306c0af8d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.333304 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce7f23-89f2-4150-b18a-19306c0af8d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:06.334935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce7f23-89f2-4150-b18a-19306c0af8d8 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:06.346381 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1187/2380] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:06.350670 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc332f6-8206-464a-bd28-f6050270c3fb admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:06.363430 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc332f6-8206-464a-bd28-f6050270c3fb admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:06.368428 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a678e8e-ad56-4d0e-8c0a-b61eabad79bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:06.369092 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a678e8e-ad56-4d0e-8c0a-b61eabad79bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.369410 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a678e8e-ad56-4d0e-8c0a-b61eabad79bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.369692 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a678e8e-ad56-4d0e-8c0a-b61eabad79bd admin admin] PATH_INFO: `/v3/projects/4b6bf55e6c5e47f3895e135c4d6dc3ff/users/a3059f5115bb4bad837daf9f18752143/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.384727 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a678e8e-ad56-4d0e-8c0a-b61eabad79bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b6bf55e6c5e47f3895e135c4d6dc3ff, user_id=a3059f5115bb4bad837daf9f18752143, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:06.387423 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a678e8e-ad56-4d0e-8c0a-b61eabad79bd admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:06.401293 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1194/2381] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:06 2022] PUT /identity/v3/projects/4b6bf55e6c5e47f3895e135c4d6dc3ff/users/a3059f5115bb4bad837daf9f18752143/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:06.406678 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3128b6-e429-418d-9632-4f4a5332ba2c None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.407037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3128b6-e429-418d-9632-4f4a5332ba2c None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.407318 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3128b6-e429-418d-9632-4f4a5332ba2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.426699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-9c3128b6-e429-418d-9632-4f4a5332ba2c None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:06.427895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-9c3128b6-e429-418d-9632-4f4a5332ba2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:06.493090 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1188/2382] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:36:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:06.502765 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a678e8e-ad56-4d0e-8c0a-b61eabad79bd admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:06.504633 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-6a678e8e-ad56-4d0e-8c0a-b61eabad79bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:06.564112 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a678e8e-ad56-4d0e-8c0a-b61eabad79bd admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:06.568036 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88869a99-a12a-4dc6-b32e-555b9f94a24f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:06.568658 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88869a99-a12a-4dc6-b32e-555b9f94a24f service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.568941 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88869a99-a12a-4dc6-b32e-555b9f94a24f service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.569218 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88869a99-a12a-4dc6-b32e-555b9f94a24f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.583272 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88869a99-a12a-4dc6-b32e-555b9f94a24f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:06.584916 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88869a99-a12a-4dc6-b32e-555b9f94a24f service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:06.595964 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1195/2383] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:07.223781 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1516e2-1ddd-486a-8f80-e0d2e9df16ad None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:07.224208 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1516e2-1ddd-486a-8f80-e0d2e9df16ad None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:07.224425 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1516e2-1ddd-486a-8f80-e0d2e9df16ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:07.246434 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-1f1516e2-1ddd-486a-8f80-e0d2e9df16ad None None] MFA Rules not processed for user `b4d4d3b4817e499196e01d1df96b4775`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:07.247686 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-1f1516e2-1ddd-486a-8f80-e0d2e9df16ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:07.339039 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1189/2384] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:36:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:11.280509 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7999e7-7735-4bfa-a06e-e56aa11ec36f None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:11.280981 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7999e7-7735-4bfa-a06e-e56aa11ec36f None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:11.280981 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7999e7-7735-4bfa-a06e-e56aa11ec36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:11.358914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-8e7999e7-7735-4bfa-a06e-e56aa11ec36f None None] MFA Rules not processed for user `a3059f5115bb4bad837daf9f18752143`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:11.360759 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8e7999e7-7735-4bfa-a06e-e56aa11ec36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:11.458160 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1196/2385] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:36:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:11.463279 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1516e2-1ddd-486a-8f80-e0d2e9df16ad None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:11.476027 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1516e2-1ddd-486a-8f80-e0d2e9df16ad None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:11.482054 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0e06ca-1d39-48a6-b175-cdd6a089531c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:11.482708 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e06ca-1d39-48a6-b175-cdd6a089531c admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:11.483190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e06ca-1d39-48a6-b175-cdd6a089531c admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:11.483466 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e06ca-1d39-48a6-b175-cdd6a089531c admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:11.484113 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0e06ca-1d39-48a6-b175-cdd6a089531c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:11.486053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0e06ca-1d39-48a6-b175-cdd6a089531c admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:11.521445 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1190/2386] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:36:11 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:11.526126 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7999e7-7735-4bfa-a06e-e56aa11ec36f None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:11.537708 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7999e7-7735-4bfa-a06e-e56aa11ec36f None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:11.556488 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:11.605025 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5753c6b1-f61a-49a4-8389-bd7ed8173a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:11.605726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753c6b1-f61a-49a4-8389-bd7ed8173a47 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:11.606006 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753c6b1-f61a-49a4-8389-bd7ed8173a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:11.606327 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753c6b1-f61a-49a4-8389-bd7ed8173a47 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:11.607024 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5753c6b1-f61a-49a4-8389-bd7ed8173a47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:11.609306 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5753c6b1-f61a-49a4-8389-bd7ed8173a47 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:11.637377 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1197/2387] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:36:11 2022] POST /identity/v3/users => generated 295 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:11.638644 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0e06ca-1d39-48a6-b175-cdd6a089531c admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:11.650137 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0e06ca-1d39-48a6-b175-cdd6a089531c admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:11.654853 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e17d46c-11fa-4757-8e79-f4700efe2b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:11.655464 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17d46c-11fa-4757-8e79-f4700efe2b6d admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:11.655751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17d46c-11fa-4757-8e79-f4700efe2b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:11.656029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17d46c-11fa-4757-8e79-f4700efe2b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:11.656838 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e17d46c-11fa-4757-8e79-f4700efe2b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:11.658673 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e17d46c-11fa-4757-8e79-f4700efe2b6d admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:11.660978 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:11.674766 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1191/2388] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:11.675807 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5753c6b1-f61a-49a4-8389-bd7ed8173a47 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:11.686162 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5753c6b1-f61a-49a4-8389-bd7ed8173a47 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:11.690924 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f072a43-426d-4855-be68-bfb5f5f0cff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:11.691607 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f072a43-426d-4855-be68-bfb5f5f0cff0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:11.691902 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f072a43-426d-4855-be68-bfb5f5f0cff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:11.692209 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f072a43-426d-4855-be68-bfb5f5f0cff0 admin admin] PATH_INFO: `/v3/projects/38a410afaff64327b9140b767b4ae48c/users/41e9711b60e4413b9df5575391dae5cb/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:11.708592 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f072a43-426d-4855-be68-bfb5f5f0cff0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38a410afaff64327b9140b767b4ae48c, user_id=41e9711b60e4413b9df5575391dae5cb, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:11.711980 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f072a43-426d-4855-be68-bfb5f5f0cff0 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:11.726785 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1198/2389] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:11 2022] PUT /identity/v3/projects/38a410afaff64327b9140b767b4ae48c/users/41e9711b60e4413b9df5575391dae5cb/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:36:11.731383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e17d46c-11fa-4757-8e79-f4700efe2b6d admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:11.742330 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e17d46c-11fa-4757-8e79-f4700efe2b6d admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:11.805392 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f08fe3-7c3f-46c4-84eb-1b109c596dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:11.806022 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f08fe3-7c3f-46c4-84eb-1b109c596dc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:11.806345 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f08fe3-7c3f-46c4-84eb-1b109c596dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:11.806623 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f08fe3-7c3f-46c4-84eb-1b109c596dc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:11.807311 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f08fe3-7c3f-46c4-84eb-1b109c596dc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:11.809177 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f08fe3-7c3f-46c4-84eb-1b109c596dc8 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:11.823892 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1192/2390] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:36:11.829471 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f072a43-426d-4855-be68-bfb5f5f0cff0 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:11.842864 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f072a43-426d-4855-be68-bfb5f5f0cff0 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:11.849128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e6261-2ddb-4b3a-98d9-a9f8f708d58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:11.849868 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e6261-2ddb-4b3a-98d9-a9f8f708d58a admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:11.850497 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e6261-2ddb-4b3a-98d9-a9f8f708d58a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:11.850951 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e6261-2ddb-4b3a-98d9-a9f8f708d58a admin admin] PATH_INFO: `/v3/projects/38a410afaff64327b9140b767b4ae48c/users/41e9711b60e4413b9df5575391dae5cb/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:11.853704 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e6261-2ddb-4b3a-98d9-a9f8f708d58a admin admin] RBAC: Authorizing `identity:create_grant(project_id=38a410afaff64327b9140b767b4ae48c, user_id=41e9711b60e4413b9df5575391dae5cb, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:11.856850 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e6261-2ddb-4b3a-98d9-a9f8f708d58a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:11.873486 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1199/2391] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:11 2022] PUT /identity/v3/projects/38a410afaff64327b9140b767b4ae48c/users/41e9711b60e4413b9df5575391dae5cb/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:14.272195 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faeef13-e293-4784-8f76-c6ce591423e3 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:14.273177 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faeef13-e293-4784-8f76-c6ce591423e3 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:14.273658 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faeef13-e293-4784-8f76-c6ce591423e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:14.308238 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-3faeef13-e293-4784-8f76-c6ce591423e3 None None] MFA Rules not processed for user `a5b3e53fd1314e17ad28c42a44e8eff3`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:14.309847 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-3faeef13-e293-4784-8f76-c6ce591423e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:14.414760 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1193/2392] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Sep 1 16:36:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:14.471694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e6261-2ddb-4b3a-98d9-a9f8f708d58a admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:14.474568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-239e6261-2ddb-4b3a-98d9-a9f8f708d58a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:14.565557 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e6261-2ddb-4b3a-98d9-a9f8f708d58a admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:14.572288 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff08de8-3383-4854-9d39-a074752c2357 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8137f6b2ced040bd9752239c7e987f03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:14.572844 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff08de8-3383-4854-9d39-a074752c2357 service swift] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:14.573281 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff08de8-3383-4854-9d39-a074752c2357 service swift] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:14.573564 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff08de8-3383-4854-9d39-a074752c2357 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:14.586654 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff08de8-3383-4854-9d39-a074752c2357 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:14.588403 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff08de8-3383-4854-9d39-a074752c2357 service swift] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:14.599695 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1200/2393] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 1 16:36:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:36:21.192994 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d27eec-7d0a-4b0e-be3a-4df134be1674 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:21.193626 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d27eec-7d0a-4b0e-be3a-4df134be1674 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:21.193626 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d27eec-7d0a-4b0e-be3a-4df134be1674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:21.225755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-72d27eec-7d0a-4b0e-be3a-4df134be1674 None None] MFA Rules not processed for user `41e9711b60e4413b9df5575391dae5cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:21.227012 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-72d27eec-7d0a-4b0e-be3a-4df134be1674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:21.320609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1194/2394] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:36:21 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:21.334900 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff08de8-3383-4854-9d39-a074752c2357 service swift] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:21.337063 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-aff08de8-3383-4854-9d39-a074752c2357 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:21.406292 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:21.425883 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff08de8-3383-4854-9d39-a074752c2357 service swift] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:21.434508 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57762761-aa6c-483b-b0f6-1876bd32f986 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:21.435802 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57762761-aa6c-483b-b0f6-1876bd32f986 service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:21.436088 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57762761-aa6c-483b-b0f6-1876bd32f986 service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:21.436362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57762761-aa6c-483b-b0f6-1876bd32f986 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:21.496105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57762761-aa6c-483b-b0f6-1876bd32f986 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:21.497627 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57762761-aa6c-483b-b0f6-1876bd32f986 service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:21.506822 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1201/2395] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:31.316417 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=89 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:31.508075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=28 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:37.389696 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d3946-506c-4031-9d0d-b2040b18a4e6 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:37.390209 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d3946-506c-4031-9d0d-b2040b18a4e6 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:37.390375 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d3946-506c-4031-9d0d-b2040b18a4e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:37.409336 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-e16d3946-506c-4031-9d0d-b2040b18a4e6 None None] MFA Rules not processed for user `8020db56bca04d61b5a6e40384c25715`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:37.410590 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-e16d3946-506c-4031-9d0d-b2040b18a4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:37.470203 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1195/2396] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:36:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:39.286660 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57762761-aa6c-483b-b0f6-1876bd32f986 service nova] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:39.288967 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-57762761-aa6c-483b-b0f6-1876bd32f986 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:39.411570 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57762761-aa6c-483b-b0f6-1876bd32f986 service nova] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:39.418066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e5e43-e9c6-43d0-9441-0b6086817f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:39.419312 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e5e43-e9c6-43d0-9441-0b6086817f77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:39.419745 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e5e43-e9c6-43d0-9441-0b6086817f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:39.420170 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e5e43-e9c6-43d0-9441-0b6086817f77 admin admin] PATH_INFO: `/v3/users/74ce31b69cd04646843587b7de26c25b` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:39.421478 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929e5e43-e9c6-43d0-9441-0b6086817f77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74ce31b69cd04646843587b7de26c25b)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:39.423573 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929e5e43-e9c6-43d0-9441-0b6086817f77 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:39.504343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-929e5e43-e9c6-43d0-9441-0b6086817f77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74ce31b69cd04646843587b7de26c25b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:39.517873 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-929e5e43-e9c6-43d0-9441-0b6086817f77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74ce31b69cd04646843587b7de26c25b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:39.519147 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-929e5e43-e9c6-43d0-9441-0b6086817f77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74ce31b69cd04646843587b7de26c25b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:39.537647 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-929e5e43-e9c6-43d0-9441-0b6086817f77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74ce31b69cd04646843587b7de26c25b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:39.539578 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-929e5e43-e9c6-43d0-9441-0b6086817f77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74ce31b69cd04646843587b7de26c25b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:39.562651 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1202/2397] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:36:39 2022] DELETE /identity/v3/users/74ce31b69cd04646843587b7de26c25b => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:39.567412 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d7d51-3bc3-4f0c-9825-1731b6c3066a None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:39.568471 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d7d51-3bc3-4f0c-9825-1731b6c3066a None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:39.568969 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d7d51-3bc3-4f0c-9825-1731b6c3066a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:39.591257 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-f56d7d51-3bc3-4f0c-9825-1731b6c3066a None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:39.592758 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-f56d7d51-3bc3-4f0c-9825-1731b6c3066a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:39.668738 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1196/2398] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:36:39.680685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e5e43-e9c6-43d0-9441-0b6086817f77 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:39.682197 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-929e5e43-e9c6-43d0-9441-0b6086817f77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:39.769279 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e5e43-e9c6-43d0-9441-0b6086817f77 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:39.774987 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed631d83-9b75-4a9d-8198-427eeb215b98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:39.775670 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed631d83-9b75-4a9d-8198-427eeb215b98 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:39.776050 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed631d83-9b75-4a9d-8198-427eeb215b98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:39.776549 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed631d83-9b75-4a9d-8198-427eeb215b98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:39.792682 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed631d83-9b75-4a9d-8198-427eeb215b98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:39.794994 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed631d83-9b75-4a9d-8198-427eeb215b98 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:39.811275 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1203/2399] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:36:40.446695 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d7d51-3bc3-4f0c-9825-1731b6c3066a None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:40.448602 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-f56d7d51-3bc3-4f0c-9825-1731b6c3066a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:40.467285 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d7d51-3bc3-4f0c-9825-1731b6c3066a None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:40.478494 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec83752-8c5d-4e7c-8a1d-900b9f27a12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:40.479574 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec83752-8c5d-4e7c-8a1d-900b9f27a12a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:40.479863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec83752-8c5d-4e7c-8a1d-900b9f27a12a admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:40.480166 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec83752-8c5d-4e7c-8a1d-900b9f27a12a admin admin] PATH_INFO: `/v3/projects/9525e7b503134f3da12368d42261e9be` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:40.481993 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec83752-8c5d-4e7c-8a1d-900b9f27a12a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9525e7b503134f3da12368d42261e9be)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:40.484504 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec83752-8c5d-4e7c-8a1d-900b9f27a12a admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:40.567767 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5ec83752-8c5d-4e7c-8a1d-900b9f27a12a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9525e7b503134f3da12368d42261e9be', 'request_id': 'req-5ec83752-8c5d-4e7c-8a1d-900b9f27a12a'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:40.582153 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5ec83752-8c5d-4e7c-8a1d-900b9f27a12a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9525e7b503134f3da12368d42261e9be', 'request_id': 'req-5ec83752-8c5d-4e7c-8a1d-900b9f27a12a'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:40.595100 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5ec83752-8c5d-4e7c-8a1d-900b9f27a12a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9525e7b503134f3da12368d42261e9be', 'request_id': 'req-5ec83752-8c5d-4e7c-8a1d-900b9f27a12a'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:40.616168 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5ec83752-8c5d-4e7c-8a1d-900b9f27a12a admin admin] The token cache is being invalidate because project 9525e7b503134f3da12368d42261e9be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:36:40.617841 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5ec83752-8c5d-4e7c-8a1d-900b9f27a12a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:40.619308 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5ec83752-8c5d-4e7c-8a1d-900b9f27a12a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:40.623791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1197/2400] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:36:40 2022] DELETE /identity/v3/projects/9525e7b503134f3da12368d42261e9be => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:40.628723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158bb4f1-f70d-4afa-b9fa-aaa90c5607af None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:40.629343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158bb4f1-f70d-4afa-b9fa-aaa90c5607af None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:40.629343 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158bb4f1-f70d-4afa-b9fa-aaa90c5607af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:40.651799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-158bb4f1-f70d-4afa-b9fa-aaa90c5607af None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:40.653011 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-158bb4f1-f70d-4afa-b9fa-aaa90c5607af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:40.673170 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:40.727747 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1204/2401] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:36:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:40.736000 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5a4e0-4be4-4b97-8e55-fb26d78ff1a2 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:40.736288 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5a4e0-4be4-4b97-8e55-fb26d78ff1a2 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:40.736563 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5a4e0-4be4-4b97-8e55-fb26d78ff1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:40.764218 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-f2f5a4e0-4be4-4b97-8e55-fb26d78ff1a2 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:40.765486 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-f2f5a4e0-4be4-4b97-8e55-fb26d78ff1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:40.778840 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1198/2402] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:36:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:40.780220 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158bb4f1-f70d-4afa-b9fa-aaa90c5607af None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:40.791811 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158bb4f1-f70d-4afa-b9fa-aaa90c5607af None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:40.796952 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b79c0-26c2-4f4f-8441-f6e702b1e84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:40.797740 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b79c0-26c2-4f4f-8441-f6e702b1e84a admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:40.798319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b79c0-26c2-4f4f-8441-f6e702b1e84a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:40.798748 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b79c0-26c2-4f4f-8441-f6e702b1e84a admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:40.799545 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7b79c0-26c2-4f4f-8441-f6e702b1e84a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:36:40.800021 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7b79c0-26c2-4f4f-8441-f6e702b1e84a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:40.803419 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7b79c0-26c2-4f4f-8441-f6e702b1e84a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:40.817342 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1205/2403] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:36:40.822321 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5a4e0-4be4-4b97-8e55-fb26d78ff1a2 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:40.831775 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5a4e0-4be4-4b97-8e55-fb26d78ff1a2 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:40.837119 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59ceae-b801-4073-80ed-274a5c0bbf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:40.837722 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59ceae-b801-4073-80ed-274a5c0bbf0e admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:40.838005 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59ceae-b801-4073-80ed-274a5c0bbf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:40.838300 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59ceae-b801-4073-80ed-274a5c0bbf0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:40.838950 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf59ceae-b801-4073-80ed-274a5c0bbf0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:40.840870 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf59ceae-b801-4073-80ed-274a5c0bbf0e admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:40.870596 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:40.884304 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1199/2404] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:36:40 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:36:40.885784 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b79c0-26c2-4f4f-8441-f6e702b1e84a admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:40.898351 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b79c0-26c2-4f4f-8441-f6e702b1e84a admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:40.975479 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922bb32-08ca-4f09-a94b-42dc3ba0897e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:40.976477 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922bb32-08ca-4f09-a94b-42dc3ba0897e admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:40.977020 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922bb32-08ca-4f09-a94b-42dc3ba0897e admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:40.977920 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922bb32-08ca-4f09-a94b-42dc3ba0897e admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:40.978866 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922bb32-08ca-4f09-a94b-42dc3ba0897e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:40.983646 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922bb32-08ca-4f09-a94b-42dc3ba0897e admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:41.016990 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1206/2405] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:36:40 2022] POST /identity/v3/users => generated 288 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:41.020834 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59ceae-b801-4073-80ed-274a5c0bbf0e admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:41.038649 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59ceae-b801-4073-80ed-274a5c0bbf0e admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:41.045163 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185195d-7be7-4493-96f0-a5806f364e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:41.045831 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185195d-7be7-4493-96f0-a5806f364e15 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:41.046229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185195d-7be7-4493-96f0-a5806f364e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:41.046525 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185195d-7be7-4493-96f0-a5806f364e15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:41.047219 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4185195d-7be7-4493-96f0-a5806f364e15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:41.049122 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4185195d-7be7-4493-96f0-a5806f364e15 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:41.065660 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1200/2406] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:41.070519 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922bb32-08ca-4f09-a94b-42dc3ba0897e admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:41.082808 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922bb32-08ca-4f09-a94b-42dc3ba0897e admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:41.089010 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac3ebc-c878-4a48-9d9e-a34fdc7d7b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:41.090289 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac3ebc-c878-4a48-9d9e-a34fdc7d7b9d admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:41.090710 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac3ebc-c878-4a48-9d9e-a34fdc7d7b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:41.091111 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac3ebc-c878-4a48-9d9e-a34fdc7d7b9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:41.091891 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac3ebc-c878-4a48-9d9e-a34fdc7d7b9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:41.093839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac3ebc-c878-4a48-9d9e-a34fdc7d7b9d admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:41.109767 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1207/2407] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:36:41.113670 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185195d-7be7-4493-96f0-a5806f364e15 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:41.125245 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185195d-7be7-4493-96f0-a5806f364e15 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:41.130684 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141afde-9692-436c-91a9-7c5bca02712b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:41.132002 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141afde-9692-436c-91a9-7c5bca02712b admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:41.132442 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141afde-9692-436c-91a9-7c5bca02712b admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:41.132974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141afde-9692-436c-91a9-7c5bca02712b admin admin] PATH_INFO: `/v3/projects/6536d1bc0651452092114f0a0650583e/users/f7f1c977d036430683b22364ab454102/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:41.154671 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9141afde-9692-436c-91a9-7c5bca02712b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6536d1bc0651452092114f0a0650583e, user_id=f7f1c977d036430683b22364ab454102, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:41.157427 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9141afde-9692-436c-91a9-7c5bca02712b admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:41.172937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1201/2408] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:41 2022] PUT /identity/v3/projects/6536d1bc0651452092114f0a0650583e/users/f7f1c977d036430683b22364ab454102/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:36:41.183836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeaba58-a200-4a62-a91a-14f7e757b786 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:41.184412 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeaba58-a200-4a62-a91a-14f7e757b786 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:41.184875 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeaba58-a200-4a62-a91a-14f7e757b786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:41.222630 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-deeaba58-a200-4a62-a91a-14f7e757b786 None None] MFA Rules not processed for user `f7f1c977d036430683b22364ab454102`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:41.224061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-deeaba58-a200-4a62-a91a-14f7e757b786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:41.327523 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1208/2409] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:36:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:36:41.340841 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141afde-9692-436c-91a9-7c5bca02712b admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:41.343538 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-9141afde-9692-436c-91a9-7c5bca02712b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:41.454797 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141afde-9692-436c-91a9-7c5bca02712b admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:41.464776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4efe7a6-b1b3-4f76-a1fc-4f146d566122 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:41.465435 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efe7a6-b1b3-4f76-a1fc-4f146d566122 service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:41.465721 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efe7a6-b1b3-4f76-a1fc-4f146d566122 service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:41.465996 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efe7a6-b1b3-4f76-a1fc-4f146d566122 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:41.478773 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4efe7a6-b1b3-4f76-a1fc-4f146d566122 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:41.483280 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4efe7a6-b1b3-4f76-a1fc-4f146d566122 service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:41.507756 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1202/2410] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:42.956887 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80907766-8766-45c0-b91e-18fd356248b1 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:42.957512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80907766-8766-45c0-b91e-18fd356248b1 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:42.957512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80907766-8766-45c0-b91e-18fd356248b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:42.979925 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-80907766-8766-45c0-b91e-18fd356248b1 None None] MFA Rules not processed for user `f7f1c977d036430683b22364ab454102`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:42.981083 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-80907766-8766-45c0-b91e-18fd356248b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:42.988563 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1209/2411] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:36:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:43.563154 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44a2bc-5ed8-4be2-a4d7-f9b6a9d209f2 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:43.563675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44a2bc-5ed8-4be2-a4d7-f9b6a9d209f2 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:43.563988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44a2bc-5ed8-4be2-a4d7-f9b6a9d209f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:43.593268 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-eb44a2bc-5ed8-4be2-a4d7-f9b6a9d209f2 None None] MFA Rules not processed for user `a5b3e53fd1314e17ad28c42a44e8eff3`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:43.595087 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-eb44a2bc-5ed8-4be2-a4d7-f9b6a9d209f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:43.668131 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1203/2412] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Sep 1 16:36:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:36:43.698993 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80907766-8766-45c0-b91e-18fd356248b1 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:43.701203 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-80907766-8766-45c0-b91e-18fd356248b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:43.728271 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:43.803355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80907766-8766-45c0-b91e-18fd356248b1 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:43.809868 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884067b-185d-4453-9c47-8825dc1d79c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8137f6b2ced040bd9752239c7e987f03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:43.810772 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884067b-185d-4453-9c47-8825dc1d79c7 service swift] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:43.811258 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884067b-185d-4453-9c47-8825dc1d79c7 service swift] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:43.811746 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884067b-185d-4453-9c47-8825dc1d79c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:43.822198 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884067b-185d-4453-9c47-8825dc1d79c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:43.826301 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884067b-185d-4453-9c47-8825dc1d79c7 service swift] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:43.846021 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1210/2413] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 1 16:36:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:36:53.659908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=66,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:53.836078 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=27 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:56.223585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be1313-1f7f-406f-a7fb-cae108fec00e None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:56.224118 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be1313-1f7f-406f-a7fb-cae108fec00e None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:56.224367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be1313-1f7f-406f-a7fb-cae108fec00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:56.369986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-a5be1313-1f7f-406f-a7fb-cae108fec00e None None] MFA Rules not processed for user `a5b3e53fd1314e17ad28c42a44e8eff3`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:56.371257 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-a5be1313-1f7f-406f-a7fb-cae108fec00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:56.394705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1204/2414] 10.222.0.18 () {58 vars in 967 bytes} [Thu Sep 1 16:36:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:56.407553 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884067b-185d-4453-9c47-8825dc1d79c7 service swift] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:56.417935 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884067b-185d-4453-9c47-8825dc1d79c7 service swift] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:56.423442 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b63c82-ffa6-4e47-9a94-ba3579c3c2db service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8137f6b2ced040bd9752239c7e987f03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:56.424049 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b63c82-ffa6-4e47-9a94-ba3579c3c2db service swift] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:56.424329 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b63c82-ffa6-4e47-9a94-ba3579c3c2db service swift] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:56.424597 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b63c82-ffa6-4e47-9a94-ba3579c3c2db service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:56.435127 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b63c82-ffa6-4e47-9a94-ba3579c3c2db service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:56.437450 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b63c82-ffa6-4e47-9a94-ba3579c3c2db service swift] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:56.461358 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1211/2415] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 1 16:36:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:36:58.956003 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be1313-1f7f-406f-a7fb-cae108fec00e None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:58.957801 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-a5be1313-1f7f-406f-a7fb-cae108fec00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:59.040757 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be1313-1f7f-406f-a7fb-cae108fec00e None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:59.045134 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef905f-bcd6-4d28-9ee0-d469aa58809d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:59.045928 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef905f-bcd6-4d28-9ee0-d469aa58809d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:59.046207 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef905f-bcd6-4d28-9ee0-d469aa58809d admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:59.046500 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef905f-bcd6-4d28-9ee0-d469aa58809d admin admin] PATH_INFO: `/v3/users/7425d9aefcf940709f3a3aad4166755c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:59.047643 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef905f-bcd6-4d28-9ee0-d469aa58809d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7425d9aefcf940709f3a3aad4166755c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:59.049676 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef905f-bcd6-4d28-9ee0-d469aa58809d admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:59.106354 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-3cef905f-bcd6-4d28-9ee0-d469aa58809d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7425d9aefcf940709f3a3aad4166755c'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:59.107582 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-3cef905f-bcd6-4d28-9ee0-d469aa58809d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7425d9aefcf940709f3a3aad4166755c'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:59.115962 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-3cef905f-bcd6-4d28-9ee0-d469aa58809d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7425d9aefcf940709f3a3aad4166755c'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:59.116779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-3cef905f-bcd6-4d28-9ee0-d469aa58809d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7425d9aefcf940709f3a3aad4166755c'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:59.132817 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-3cef905f-bcd6-4d28-9ee0-d469aa58809d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7425d9aefcf940709f3a3aad4166755c'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:59.148710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1205/2416] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:36:58 2022] DELETE /identity/v3/users/7425d9aefcf940709f3a3aad4166755c => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:59.513237 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b63c82-ffa6-4e47-9a94-ba3579c3c2db service swift] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:59.532151 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b63c82-ffa6-4e47-9a94-ba3579c3c2db service swift] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:59.600669 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39a310-83ef-4fc8-a105-f80127217679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:59.601317 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39a310-83ef-4fc8-a105-f80127217679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:59.601601 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39a310-83ef-4fc8-a105-f80127217679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:59.602122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39a310-83ef-4fc8-a105-f80127217679 admin admin] PATH_INFO: `/v3/projects/0939af057f4940298089c8f89ec2865f` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:59.603297 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad39a310-83ef-4fc8-a105-f80127217679 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0939af057f4940298089c8f89ec2865f)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:59.605236 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad39a310-83ef-4fc8-a105-f80127217679 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:59.655967 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-ad39a310-83ef-4fc8-a105-f80127217679 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0939af057f4940298089c8f89ec2865f', 'request_id': 'req-ad39a310-83ef-4fc8-a105-f80127217679'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:59.662141 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-ad39a310-83ef-4fc8-a105-f80127217679 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0939af057f4940298089c8f89ec2865f', 'request_id': 'req-ad39a310-83ef-4fc8-a105-f80127217679'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:59.670791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-ad39a310-83ef-4fc8-a105-f80127217679 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0939af057f4940298089c8f89ec2865f', 'request_id': 'req-ad39a310-83ef-4fc8-a105-f80127217679'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:59.692427 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-ad39a310-83ef-4fc8-a105-f80127217679 admin admin] The token cache is being invalidate because project 0939af057f4940298089c8f89ec2865f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:36:59.692733 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-ad39a310-83ef-4fc8-a105-f80127217679 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:59.693394 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-ad39a310-83ef-4fc8-a105-f80127217679 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:59.695178 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1212/2417] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:36:59 2022] DELETE /identity/v3/projects/0939af057f4940298089c8f89ec2865f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:00.549776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef905f-bcd6-4d28-9ee0-d469aa58809d admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:00.551842 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-3cef905f-bcd6-4d28-9ee0-d469aa58809d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:00.569132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef905f-bcd6-4d28-9ee0-d469aa58809d admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:00.671946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b651f-7a7a-4b29-9e8e-3652754c0551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:00.672390 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b651f-7a7a-4b29-9e8e-3652754c0551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:00.672802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b651f-7a7a-4b29-9e8e-3652754c0551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:00.673077 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b651f-7a7a-4b29-9e8e-3652754c0551 admin admin] PATH_INFO: `/v3/users/13193ad3a22244279f9e2e93a2e9ac16` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:00.674224 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549b651f-7a7a-4b29-9e8e-3652754c0551 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13193ad3a22244279f9e2e93a2e9ac16)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:00.676216 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549b651f-7a7a-4b29-9e8e-3652754c0551 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:00.690297 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:00.744587 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-549b651f-7a7a-4b29-9e8e-3652754c0551 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13193ad3a22244279f9e2e93a2e9ac16'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:00.745296 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-549b651f-7a7a-4b29-9e8e-3652754c0551 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13193ad3a22244279f9e2e93a2e9ac16'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:00.758112 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-549b651f-7a7a-4b29-9e8e-3652754c0551 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13193ad3a22244279f9e2e93a2e9ac16'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:00.759405 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-549b651f-7a7a-4b29-9e8e-3652754c0551 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13193ad3a22244279f9e2e93a2e9ac16'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:00.774671 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-549b651f-7a7a-4b29-9e8e-3652754c0551 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13193ad3a22244279f9e2e93a2e9ac16'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:00.790932 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1206/2418] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:37:00 2022] DELETE /identity/v3/users/13193ad3a22244279f9e2e93a2e9ac16 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:00.795435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ee2b9-16dd-4c19-941c-e31aae271814 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:00.796344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ee2b9-16dd-4c19-941c-e31aae271814 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:00.796687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ee2b9-16dd-4c19-941c-e31aae271814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:00.817084 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-db1ee2b9-16dd-4c19-941c-e31aae271814 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:00.819555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-db1ee2b9-16dd-4c19-941c-e31aae271814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:00.891317 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1213/2419] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:37:00.901066 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b651f-7a7a-4b29-9e8e-3652754c0551 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:00.903197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-549b651f-7a7a-4b29-9e8e-3652754c0551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:01.022137 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b651f-7a7a-4b29-9e8e-3652754c0551 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:01.027759 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469b23c-2781-4cf2-9acb-ebc9745b6a85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:01.029332 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469b23c-2781-4cf2-9acb-ebc9745b6a85 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:01.029728 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469b23c-2781-4cf2-9acb-ebc9745b6a85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:01.030629 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469b23c-2781-4cf2-9acb-ebc9745b6a85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:01.065549 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0469b23c-2781-4cf2-9acb-ebc9745b6a85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:01.068331 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0469b23c-2781-4cf2-9acb-ebc9745b6a85 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:01.087006 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1207/2420] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:37:01.427434 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d893ab-3fc0-4ec4-bb5b-a57f7b269278 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:01.427930 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d893ab-3fc0-4ec4-bb5b-a57f7b269278 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:01.431167 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d893ab-3fc0-4ec4-bb5b-a57f7b269278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:01.435568 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469b23c-2781-4cf2-9acb-ebc9745b6a85 service neutron] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:01.437434 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-0469b23c-2781-4cf2-9acb-ebc9745b6a85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:01.460745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469b23c-2781-4cf2-9acb-ebc9745b6a85 service neutron] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:01.467495 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca758a-673b-463b-acc9-f030fcee527e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:01.471046 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca758a-673b-463b-acc9-f030fcee527e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:01.471827 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca758a-673b-463b-acc9-f030fcee527e admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:01.472358 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca758a-673b-463b-acc9-f030fcee527e admin admin] PATH_INFO: `/v3/projects/fa7fc5a55f4a41e28aaa945a72088725` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:01.473911 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-f9d893ab-3fc0-4ec4-bb5b-a57f7b269278 None None] MFA Rules not processed for user `a3059f5115bb4bad837daf9f18752143`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:01.474042 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca758a-673b-463b-acc9-f030fcee527e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa7fc5a55f4a41e28aaa945a72088725)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:01.476024 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f9d893ab-3fc0-4ec4-bb5b-a57f7b269278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:01.476634 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca758a-673b-463b-acc9-f030fcee527e admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:01.493822 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:01.547315 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-12ca758a-673b-463b-acc9-f030fcee527e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa7fc5a55f4a41e28aaa945a72088725', 'request_id': 'req-12ca758a-673b-463b-acc9-f030fcee527e'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:01.556213 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-12ca758a-673b-463b-acc9-f030fcee527e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa7fc5a55f4a41e28aaa945a72088725', 'request_id': 'req-12ca758a-673b-463b-acc9-f030fcee527e'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:01.563698 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-12ca758a-673b-463b-acc9-f030fcee527e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa7fc5a55f4a41e28aaa945a72088725', 'request_id': 'req-12ca758a-673b-463b-acc9-f030fcee527e'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:01.578290 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1214/2421] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:37:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:01.585046 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-12ca758a-673b-463b-acc9-f030fcee527e admin admin] The token cache is being invalidate because project fa7fc5a55f4a41e28aaa945a72088725 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:37:01.585553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-12ca758a-673b-463b-acc9-f030fcee527e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:01.586599 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-12ca758a-673b-463b-acc9-f030fcee527e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:01.590539 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1208/2422] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:37:01 2022] DELETE /identity/v3/projects/fa7fc5a55f4a41e28aaa945a72088725 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:07.360616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2785f5-306d-4057-b9e3-9d515be4cd3b None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:07.361180 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2785f5-306d-4057-b9e3-9d515be4cd3b None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:07.361317 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2785f5-306d-4057-b9e3-9d515be4cd3b None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:07.362625 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1215/2423] 10.222.0.18 () {54 vars in 921 bytes} [Thu Sep 1 16:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:37:07.370015 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f598e-6d2b-4775-a61a-a505631ee714 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:07.370482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f598e-6d2b-4775-a61a-a505631ee714 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:07.370685 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f598e-6d2b-4775-a61a-a505631ee714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:07.395879 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-7b4f598e-6d2b-4775-a61a-a505631ee714 None None] MFA Rules not processed for user `278e6d6aed414c758fe3e9f4df6bf81e`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:07.397904 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-7b4f598e-6d2b-4775-a61a-a505631ee714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:07.410704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:07.470510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1209/2424] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Sep 1 16:37:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:37:07.482326 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2785f5-306d-4057-b9e3-9d515be4cd3b None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:07.484186 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3d2785f5-306d-4057-b9e3-9d515be4cd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:07.542559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2785f5-306d-4057-b9e3-9d515be4cd3b None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:07.546059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8ac33-9158-4731-befa-2360c0f0b7e6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b09f6d550c9142bf956b225bf5172e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:07.546696 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8ac33-9158-4731-befa-2360c0f0b7e6 service glance] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:07.546979 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8ac33-9158-4731-befa-2360c0f0b7e6 service glance] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:07.547251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8ac33-9158-4731-befa-2360c0f0b7e6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:07.557714 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f8ac33-9158-4731-befa-2360c0f0b7e6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:07.559201 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f8ac33-9158-4731-befa-2360c0f0b7e6 service glance] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:07.570531 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1216/2425] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:07.638176 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59821ce9-ef66-477a-ab4f-0b8654963aba None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:07.638491 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59821ce9-ef66-477a-ab4f-0b8654963aba None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:07.638772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59821ce9-ef66-477a-ab4f-0b8654963aba None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:07.640127 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1210/2426] 10.222.0.18 () {54 vars in 921 bytes} [Thu Sep 1 16:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:37:07.644751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d83e80-b677-4ba9-ae75-af50f0631239 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:07.645039 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d83e80-b677-4ba9-ae75-af50f0631239 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:07.645318 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d83e80-b677-4ba9-ae75-af50f0631239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:07.664608 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-f4d83e80-b677-4ba9-ae75-af50f0631239 None None] MFA Rules not processed for user `ffa0f2160a934906b59e6f845edb0776`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:07.665879 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f4d83e80-b677-4ba9-ae75-af50f0631239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:07.716213 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1217/2427] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Sep 1 16:37:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:07.728125 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59821ce9-ef66-477a-ab4f-0b8654963aba None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:07.729767 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-59821ce9-ef66-477a-ab4f-0b8654963aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:07.743080 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59821ce9-ef66-477a-ab4f-0b8654963aba None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:07.746567 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f0adf-fd91-43a5-beaf-ee1be7747092 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:07.747147 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f0adf-fd91-43a5-beaf-ee1be7747092 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:07.747424 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f0adf-fd91-43a5-beaf-ee1be7747092 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:07.747696 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f0adf-fd91-43a5-beaf-ee1be7747092 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:07.754496 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f0adf-fd91-43a5-beaf-ee1be7747092 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:07.755923 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f0adf-fd91-43a5-beaf-ee1be7747092 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:07.764527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1211/2428] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:07.802848 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d83e80-b677-4ba9-ae75-af50f0631239 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:07.805583 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f4d83e80-b677-4ba9-ae75-af50f0631239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:07.893680 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980949c-e576-4ada-b433-aaa6d3362578 None None] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:07.893968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980949c-e576-4ada-b433-aaa6d3362578 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:07.894257 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980949c-e576-4ada-b433-aaa6d3362578 None None] PATH_INFO: `/` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:07.896906 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1212/2429] 10.222.0.18 () {54 vars in 921 bytes} [Thu Sep 1 16:37:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:37:07.902447 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde099a-2cb6-4ff7-b024-bfe7136bd630 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:07.902892 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde099a-2cb6-4ff7-b024-bfe7136bd630 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:07.903279 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde099a-2cb6-4ff7-b024-bfe7136bd630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:07.903470 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d83e80-b677-4ba9-ae75-af50f0631239 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:07.909618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f7d2d7-51d8-421e-a5d4-5c18085a5189 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ddfb151bbf844efae73ec1990e7eaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:07.910265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7d2d7-51d8-421e-a5d4-5c18085a5189 service placement] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:07.910551 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7d2d7-51d8-421e-a5d4-5c18085a5189 service placement] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:07.910827 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7d2d7-51d8-421e-a5d4-5c18085a5189 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:07.913171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a1f7d2d7-51d8-421e-a5d4-5c18085a5189 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:07.927284 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-9dde099a-2cb6-4ff7-b024-bfe7136bd630 None None] MFA Rules not processed for user `278e6d6aed414c758fe3e9f4df6bf81e`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:07.928923 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f7d2d7-51d8-421e-a5d4-5c18085a5189 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:07.929309 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-9dde099a-2cb6-4ff7-b024-bfe7136bd630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:07.931292 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f7d2d7-51d8-421e-a5d4-5c18085a5189 service placement] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:07.943465 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1218/2430] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:07.943633 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1213/2431] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Sep 1 16:37:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:37:07.957562 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f7d2d7-51d8-421e-a5d4-5c18085a5189 service placement] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:07.965285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:07.969846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f7d2d7-51d8-421e-a5d4-5c18085a5189 service placement] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:07.974337 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb2d64-698a-4c23-959e-937e451abcf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:07.974893 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb2d64-698a-4c23-959e-937e451abcf6 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:07.975925 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb2d64-698a-4c23-959e-937e451abcf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:07.976212 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb2d64-698a-4c23-959e-937e451abcf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:07.987041 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bb2d64-698a-4c23-959e-937e451abcf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:07.988476 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bb2d64-698a-4c23-959e-937e451abcf6 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:07.999948 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1219/2432] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:11.903418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae4db7-81c4-4f86-a311-4d90e3860980 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:11.904545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae4db7-81c4-4f86-a311-4d90e3860980 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:11.905052 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae4db7-81c4-4f86-a311-4d90e3860980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:11.938097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-c3ae4db7-81c4-4f86-a311-4d90e3860980 None None] MFA Rules not processed for user `a5b3e53fd1314e17ad28c42a44e8eff3`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:11.941100 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-c3ae4db7-81c4-4f86-a311-4d90e3860980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:12.047692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1214/2433] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Sep 1 16:37:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:37:12.062575 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb2d64-698a-4c23-959e-937e451abcf6 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:12.064066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-63bb2d64-698a-4c23-959e-937e451abcf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:12.143085 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb2d64-698a-4c23-959e-937e451abcf6 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:12.164099 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5097d27-afb8-4c3e-9c53-1912f289e311 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8137f6b2ced040bd9752239c7e987f03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:12.164815 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5097d27-afb8-4c3e-9c53-1912f289e311 service swift] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:12.165101 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5097d27-afb8-4c3e-9c53-1912f289e311 service swift] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:12.165380 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5097d27-afb8-4c3e-9c53-1912f289e311 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:12.184378 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5097d27-afb8-4c3e-9c53-1912f289e311 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:12.190188 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5097d27-afb8-4c3e-9c53-1912f289e311 service swift] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:12.211241 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1220/2434] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 1 16:37:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:37:12.883727 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2122c75-6e98-48c6-9551-776074c8ff45 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:12.884043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2122c75-6e98-48c6-9551-776074c8ff45 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:12.884319 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2122c75-6e98-48c6-9551-776074c8ff45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:12.904800 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-b2122c75-6e98-48c6-9551-776074c8ff45 None None] MFA Rules not processed for user `a5b3e53fd1314e17ad28c42a44e8eff3`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:12.906025 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-b2122c75-6e98-48c6-9551-776074c8ff45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:12.916223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1215/2435] 10.222.0.18 () {58 vars in 967 bytes} [Thu Sep 1 16:37:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:12.942828 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5097d27-afb8-4c3e-9c53-1912f289e311 service swift] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:12.976438 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5097d27-afb8-4c3e-9c53-1912f289e311 service swift] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:12.982963 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9f1e2-887e-4a00-bd60-3891e3e23ca5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8137f6b2ced040bd9752239c7e987f03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:12.984531 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9f1e2-887e-4a00-bd60-3891e3e23ca5 service swift] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:12.985015 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9f1e2-887e-4a00-bd60-3891e3e23ca5 service swift] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:12.985526 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9f1e2-887e-4a00-bd60-3891e3e23ca5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:12.995887 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc9f1e2-887e-4a00-bd60-3891e3e23ca5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:12.997917 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc9f1e2-887e-4a00-bd60-3891e3e23ca5 service swift] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:13.012905 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1221/2436] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 1 16:37:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:37:13.273646 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5ab65-578c-4e4e-b3e4-813efec529a7 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:13.273982 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5ab65-578c-4e4e-b3e4-813efec529a7 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:13.274210 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5ab65-578c-4e4e-b3e4-813efec529a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:13.294386 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-97e5ab65-578c-4e4e-b3e4-813efec529a7 None None] MFA Rules not processed for user `a5b3e53fd1314e17ad28c42a44e8eff3`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:13.295579 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-97e5ab65-578c-4e4e-b3e4-813efec529a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:13.303032 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1216/2437] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Sep 1 16:37:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:13.316968 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9f1e2-887e-4a00-bd60-3891e3e23ca5 service swift] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:13.359047 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9f1e2-887e-4a00-bd60-3891e3e23ca5 service swift] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:13.371937 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9a6eee-2616-488b-9e04-3bd146bc4e75 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8137f6b2ced040bd9752239c7e987f03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:13.372878 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a6eee-2616-488b-9e04-3bd146bc4e75 service swift] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:13.373320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a6eee-2616-488b-9e04-3bd146bc4e75 service swift] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:13.373738 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a6eee-2616-488b-9e04-3bd146bc4e75 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:13.386210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9a6eee-2616-488b-9e04-3bd146bc4e75 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:13.388028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9a6eee-2616-488b-9e04-3bd146bc4e75 service swift] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:13.398564 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1222/2438] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 1 16:37:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:37:13.794759 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f45d6c-1141-4060-be5d-f4cde35d18c1 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:13.795080 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f45d6c-1141-4060-be5d-f4cde35d18c1 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:13.795357 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f45d6c-1141-4060-be5d-f4cde35d18c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:13.814389 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-e2f45d6c-1141-4060-be5d-f4cde35d18c1 None None] MFA Rules not processed for user `a5b3e53fd1314e17ad28c42a44e8eff3`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:13.815582 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-e2f45d6c-1141-4060-be5d-f4cde35d18c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:13.824009 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1217/2439] 10.222.0.18 () {58 vars in 967 bytes} [Thu Sep 1 16:37:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:13.835368 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9a6eee-2616-488b-9e04-3bd146bc4e75 service swift] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:13.846637 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9a6eee-2616-488b-9e04-3bd146bc4e75 service swift] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:13.852623 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2bbf0c-4b21-4f6f-876e-8e77db466eb7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8137f6b2ced040bd9752239c7e987f03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:13.853295 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bbf0c-4b21-4f6f-876e-8e77db466eb7 service swift] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:13.853666 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bbf0c-4b21-4f6f-876e-8e77db466eb7 service swift] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:13.854006 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bbf0c-4b21-4f6f-876e-8e77db466eb7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:13.862426 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2bbf0c-4b21-4f6f-876e-8e77db466eb7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:13.864021 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2bbf0c-4b21-4f6f-876e-8e77db466eb7 service swift] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:13.872274 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1223/2440] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 1 16:37:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 16:37:15.048987 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086af25-a06a-449d-b55b-4d105d4f3c90 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:15.049446 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086af25-a06a-449d-b55b-4d105d4f3c90 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:15.049542 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086af25-a06a-449d-b55b-4d105d4f3c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:15.069377 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-3086af25-a06a-449d-b55b-4d105d4f3c90 None None] MFA Rules not processed for user `a5b3e53fd1314e17ad28c42a44e8eff3`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:15.070607 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-3086af25-a06a-449d-b55b-4d105d4f3c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:15.077643 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1218/2441] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Sep 1 16:37:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:15.099079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2bbf0c-4b21-4f6f-876e-8e77db466eb7 service swift] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:15.109988 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2bbf0c-4b21-4f6f-876e-8e77db466eb7 service swift] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:15.115015 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df42f84-b51c-4c4e-ad55-553cb05ef0b2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8137f6b2ced040bd9752239c7e987f03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:15.116155 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df42f84-b51c-4c4e-ad55-553cb05ef0b2 service swift] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:15.116435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df42f84-b51c-4c4e-ad55-553cb05ef0b2 service swift] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:15.116707 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df42f84-b51c-4c4e-ad55-553cb05ef0b2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:15.124820 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df42f84-b51c-4c4e-ad55-553cb05ef0b2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:15.126431 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df42f84-b51c-4c4e-ad55-553cb05ef0b2 service swift] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:15.135066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1224/2442] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 1 16:37:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:37:15.999930 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086af25-a06a-449d-b55b-4d105d4f3c90 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:16.001847 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-3086af25-a06a-449d-b55b-4d105d4f3c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:16.101386 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086af25-a06a-449d-b55b-4d105d4f3c90 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:16.106340 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80dfeb9-4c31-40cc-b255-a6c7d730b6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:16.106986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dfeb9-4c31-40cc-b255-a6c7d730b6ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:16.107285 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dfeb9-4c31-40cc-b255-a6c7d730b6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:16.107548 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dfeb9-4c31-40cc-b255-a6c7d730b6ab admin admin] PATH_INFO: `/v3/users/f7f1c977d036430683b22364ab454102` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:16.108705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80dfeb9-4c31-40cc-b255-a6c7d730b6ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7f1c977d036430683b22364ab454102)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:16.110655 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80dfeb9-4c31-40cc-b255-a6c7d730b6ab admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:16.122342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:16.160120 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-a80dfeb9-4c31-40cc-b255-a6c7d730b6ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7f1c977d036430683b22364ab454102'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:16.160756 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-a80dfeb9-4c31-40cc-b255-a6c7d730b6ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7f1c977d036430683b22364ab454102'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:16.169527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-a80dfeb9-4c31-40cc-b255-a6c7d730b6ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7f1c977d036430683b22364ab454102'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:16.170187 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-a80dfeb9-4c31-40cc-b255-a6c7d730b6ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7f1c977d036430683b22364ab454102'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:16.184021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-a80dfeb9-4c31-40cc-b255-a6c7d730b6ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7f1c977d036430683b22364ab454102'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:16.196691 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1219/2443] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:37:15 2022] DELETE /identity/v3/users/f7f1c977d036430683b22364ab454102 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:16.201511 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261ff0b-8905-44e1-ad58-afe3e06452c6 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:16.202059 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261ff0b-8905-44e1-ad58-afe3e06452c6 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:16.202167 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261ff0b-8905-44e1-ad58-afe3e06452c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:16.222035 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-3261ff0b-8905-44e1-ad58-afe3e06452c6 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:16.223566 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3261ff0b-8905-44e1-ad58-afe3e06452c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:16.290129 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1225/2444] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:37:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:37:16.300096 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80dfeb9-4c31-40cc-b255-a6c7d730b6ab admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:16.301650 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-a80dfeb9-4c31-40cc-b255-a6c7d730b6ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:16.373832 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80dfeb9-4c31-40cc-b255-a6c7d730b6ab admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:16.377477 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac846bf-27ce-4c4f-ac1e-7e0866a791bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:16.378118 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac846bf-27ce-4c4f-ac1e-7e0866a791bb service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:16.378439 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac846bf-27ce-4c4f-ac1e-7e0866a791bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:16.378710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac846bf-27ce-4c4f-ac1e-7e0866a791bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:16.388282 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac846bf-27ce-4c4f-ac1e-7e0866a791bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:16.389729 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac846bf-27ce-4c4f-ac1e-7e0866a791bb service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:16.399954 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1220/2445] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:16.592084 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261ff0b-8905-44e1-ad58-afe3e06452c6 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:16.593498 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3261ff0b-8905-44e1-ad58-afe3e06452c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:16.623284 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261ff0b-8905-44e1-ad58-afe3e06452c6 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:16.628608 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dce0b5-4881-41fc-85cd-0edcc48342f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:16.629260 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dce0b5-4881-41fc-85cd-0edcc48342f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:16.629538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dce0b5-4881-41fc-85cd-0edcc48342f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:16.629812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dce0b5-4881-41fc-85cd-0edcc48342f0 admin admin] PATH_INFO: `/v3/projects/6536d1bc0651452092114f0a0650583e` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:16.630979 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dce0b5-4881-41fc-85cd-0edcc48342f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6536d1bc0651452092114f0a0650583e)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:16.633304 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dce0b5-4881-41fc-85cd-0edcc48342f0 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:16.663902 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:16.695459 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f1dce0b5-4881-41fc-85cd-0edcc48342f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6536d1bc0651452092114f0a0650583e', 'request_id': 'req-f1dce0b5-4881-41fc-85cd-0edcc48342f0'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:16.703363 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f1dce0b5-4881-41fc-85cd-0edcc48342f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6536d1bc0651452092114f0a0650583e', 'request_id': 'req-f1dce0b5-4881-41fc-85cd-0edcc48342f0'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:16.712398 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f1dce0b5-4881-41fc-85cd-0edcc48342f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6536d1bc0651452092114f0a0650583e', 'request_id': 'req-f1dce0b5-4881-41fc-85cd-0edcc48342f0'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:16.723967 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f1dce0b5-4881-41fc-85cd-0edcc48342f0 admin admin] The token cache is being invalidate because project 6536d1bc0651452092114f0a0650583e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:37:16.724265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f1dce0b5-4881-41fc-85cd-0edcc48342f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:16.724984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f1dce0b5-4881-41fc-85cd-0edcc48342f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:16.730321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1226/2446] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:37:16 2022] DELETE /identity/v3/projects/6536d1bc0651452092114f0a0650583e => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:26.395091 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:DELETE=10,keystone:SELECT=48,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:26.720738 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=15,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:42.120556 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac846bf-27ce-4c4f-ac1e-7e0866a791bb service neutron] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:42.129675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac846bf-27ce-4c4f-ac1e-7e0866a791bb service neutron] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:42.182899 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663715b2-37b5-4e55-ae31-29c32290f409 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:42.183937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663715b2-37b5-4e55-ae31-29c32290f409 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:42.184132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663715b2-37b5-4e55-ae31-29c32290f409 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:42.184397 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663715b2-37b5-4e55-ae31-29c32290f409 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:42.186135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-663715b2-37b5-4e55-ae31-29c32290f409 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:42.249667 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663715b2-37b5-4e55-ae31-29c32290f409 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:42.251644 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663715b2-37b5-4e55-ae31-29c32290f409 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:42.261444 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1221/2447] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:44.577784 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099dc00-a993-43fe-83f3-7044aa056ccc None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:44.578276 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099dc00-a993-43fe-83f3-7044aa056ccc None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:44.578396 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099dc00-a993-43fe-83f3-7044aa056ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:44.599979 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-d099dc00-a993-43fe-83f3-7044aa056ccc None None] MFA Rules not processed for user `a5b3e53fd1314e17ad28c42a44e8eff3`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:44.601486 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d099dc00-a993-43fe-83f3-7044aa056ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:44.660955 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1227/2448] 10.222.0.18 () {58 vars in 967 bytes} [Thu Sep 1 16:37:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:44.674562 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663715b2-37b5-4e55-ae31-29c32290f409 service neutron] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:44.676268 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-663715b2-37b5-4e55-ae31-29c32290f409 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:44.746972 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663715b2-37b5-4e55-ae31-29c32290f409 service neutron] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:44.751800 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29cc32-0a4a-4e3a-acf5-1539a206047a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8137f6b2ced040bd9752239c7e987f03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:44.752404 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29cc32-0a4a-4e3a-acf5-1539a206047a service swift] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:44.752683 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29cc32-0a4a-4e3a-acf5-1539a206047a service swift] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:44.752958 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29cc32-0a4a-4e3a-acf5-1539a206047a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:44.763461 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c29cc32-0a4a-4e3a-acf5-1539a206047a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:44.765142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c29cc32-0a4a-4e3a-acf5-1539a206047a service swift] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:44.775655 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1222/2449] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 1 16:37:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:37:54.653479 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=25 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:54.772258 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=82 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:58.445738 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f985e0-5f36-48f0-9702-3a94abafff1a None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:58.446220 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f985e0-5f36-48f0-9702-3a94abafff1a None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:58.446445 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f985e0-5f36-48f0-9702-3a94abafff1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:58.464987 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-f7f985e0-5f36-48f0-9702-3a94abafff1a None None] MFA Rules not processed for user `b4d4d3b4817e499196e01d1df96b4775`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:58.466213 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f7f985e0-5f36-48f0-9702-3a94abafff1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:58.526564 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1228/2450] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:37:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:08.518510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=25 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:44.328175 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54223c-f3d5-4886-888c-4b8858c416f8 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:44.328999 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54223c-f3d5-4886-888c-4b8858c416f8 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:44.328999 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54223c-f3d5-4886-888c-4b8858c416f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:44.356603 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-4c54223c-f3d5-4886-888c-4b8858c416f8 None None] MFA Rules not processed for user `8020db56bca04d61b5a6e40384c25715`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:44.360123 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-4c54223c-f3d5-4886-888c-4b8858c416f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:44.429239 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1223/2451] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:38:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:48.122621 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e83d06a-6bb7-47e8-b610-85a0dd0be1fd None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:48.123128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e83d06a-6bb7-47e8-b610-85a0dd0be1fd None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:48.123128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e83d06a-6bb7-47e8-b610-85a0dd0be1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:48.143076 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-2e83d06a-6bb7-47e8-b610-85a0dd0be1fd None None] MFA Rules not processed for user `8388303f9098497bb559b3bcb59fab52`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:48.144299 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2e83d06a-6bb7-47e8-b610-85a0dd0be1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:48.200733 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1229/2452] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:38:48 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:38:54.421666 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=25 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:55.151209 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54223c-f3d5-4886-888c-4b8858c416f8 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:55.152604 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-4c54223c-f3d5-4886-888c-4b8858c416f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:55.164176 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54223c-f3d5-4886-888c-4b8858c416f8 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:55.167643 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b23727-1468-40bf-ab82-af68a878557d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:55.168238 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b23727-1468-40bf-ab82-af68a878557d service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:55.168517 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b23727-1468-40bf-ab82-af68a878557d service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:55.168787 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b23727-1468-40bf-ab82-af68a878557d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:55.170504 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-87b23727-1468-40bf-ab82-af68a878557d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:55.241699 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b23727-1468-40bf-ab82-af68a878557d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:55.243173 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b23727-1468-40bf-ab82-af68a878557d service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:55.254709 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1224/2453] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:56.428842 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e83d06a-6bb7-47e8-b610-85a0dd0be1fd None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:56.430219 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2e83d06a-6bb7-47e8-b610-85a0dd0be1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:56.441950 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e83d06a-6bb7-47e8-b610-85a0dd0be1fd None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:56.445473 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde4d63-db86-4bfc-a299-58ce6dcdb197 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:56.446080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde4d63-db86-4bfc-a299-58ce6dcdb197 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:56.446405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde4d63-db86-4bfc-a299-58ce6dcdb197 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:56.446675 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde4d63-db86-4bfc-a299-58ce6dcdb197 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:56.448395 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-adde4d63-db86-4bfc-a299-58ce6dcdb197 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:56.462630 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde4d63-db86-4bfc-a299-58ce6dcdb197 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:56.464050 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde4d63-db86-4bfc-a299-58ce6dcdb197 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:56.476049 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1230/2454] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:39:05.249502 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=34 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:06.470830 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=31 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:07.224654 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b23727-1468-40bf-ab82-af68a878557d service neutron] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:07.226265 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-87b23727-1468-40bf-ab82-af68a878557d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:07.291014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b23727-1468-40bf-ab82-af68a878557d service neutron] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:07.300756 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75630a5d-d0ea-4605-9f32-b74d4a821b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:07.301395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75630a5d-d0ea-4605-9f32-b74d4a821b42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:07.301670 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75630a5d-d0ea-4605-9f32-b74d4a821b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:07.301939 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75630a5d-d0ea-4605-9f32-b74d4a821b42 admin admin] PATH_INFO: `/v3/users/b4d4d3b4817e499196e01d1df96b4775` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:07.303084 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75630a5d-d0ea-4605-9f32-b74d4a821b42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4d4d3b4817e499196e01d1df96b4775)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:07.304997 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75630a5d-d0ea-4605-9f32-b74d4a821b42 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:07.355579 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-75630a5d-d0ea-4605-9f32-b74d4a821b42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4d4d3b4817e499196e01d1df96b4775'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:07.356266 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-75630a5d-d0ea-4605-9f32-b74d4a821b42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4d4d3b4817e499196e01d1df96b4775'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:07.367035 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-75630a5d-d0ea-4605-9f32-b74d4a821b42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4d4d3b4817e499196e01d1df96b4775'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:07.367785 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-75630a5d-d0ea-4605-9f32-b74d4a821b42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4d4d3b4817e499196e01d1df96b4775'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:07.381287 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-75630a5d-d0ea-4605-9f32-b74d4a821b42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4d4d3b4817e499196e01d1df96b4775'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:07.394201 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1225/2455] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:39:07 2022] DELETE /identity/v3/users/b4d4d3b4817e499196e01d1df96b4775 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:39:07.571645 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde4d63-db86-4bfc-a299-58ce6dcdb197 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:07.582472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde4d63-db86-4bfc-a299-58ce6dcdb197 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:07.646190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6a110-4b6f-4bd8-8292-30a23f37770d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:07.646851 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6a110-4b6f-4bd8-8292-30a23f37770d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:07.647138 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6a110-4b6f-4bd8-8292-30a23f37770d admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:07.647418 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6a110-4b6f-4bd8-8292-30a23f37770d admin admin] PATH_INFO: `/v3/projects/09b1f2c2ff4e4338acd2a3367a4fdee5` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:07.648542 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6a110-4b6f-4bd8-8292-30a23f37770d admin admin] RBAC: Authorizing `identity:delete_project(project_id=09b1f2c2ff4e4338acd2a3367a4fdee5)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:07.650483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6a110-4b6f-4bd8-8292-30a23f37770d admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:07.697301 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-6ac6a110-4b6f-4bd8-8292-30a23f37770d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09b1f2c2ff4e4338acd2a3367a4fdee5', 'request_id': 'req-6ac6a110-4b6f-4bd8-8292-30a23f37770d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:07.704053 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-6ac6a110-4b6f-4bd8-8292-30a23f37770d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09b1f2c2ff4e4338acd2a3367a4fdee5', 'request_id': 'req-6ac6a110-4b6f-4bd8-8292-30a23f37770d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:07.711635 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-6ac6a110-4b6f-4bd8-8292-30a23f37770d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09b1f2c2ff4e4338acd2a3367a4fdee5', 'request_id': 'req-6ac6a110-4b6f-4bd8-8292-30a23f37770d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:07.722487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-6ac6a110-4b6f-4bd8-8292-30a23f37770d admin admin] The token cache is being invalidate because project 09b1f2c2ff4e4338acd2a3367a4fdee5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:39:07.722796 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-6ac6a110-4b6f-4bd8-8292-30a23f37770d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:07.723413 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-6ac6a110-4b6f-4bd8-8292-30a23f37770d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:07.725134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1231/2456] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:39:07 2022] DELETE /identity/v3/projects/09b1f2c2ff4e4338acd2a3367a4fdee5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:39:07.729968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7cc3c6-114e-4365-9f8f-6926a23af69b None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:07.730284 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7cc3c6-114e-4365-9f8f-6926a23af69b None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:07.730564 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7cc3c6-114e-4365-9f8f-6926a23af69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:07.749938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-6f7cc3c6-114e-4365-9f8f-6926a23af69b None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:07.751179 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-6f7cc3c6-114e-4365-9f8f-6926a23af69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:07.810070 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1226/2457] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:39:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:39:07.818685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef8a70-9950-4b47-9a37-c5fe6fbde415 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:07.818977 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef8a70-9950-4b47-9a37-c5fe6fbde415 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:07.819253 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef8a70-9950-4b47-9a37-c5fe6fbde415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:07.840319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-32ef8a70-9950-4b47-9a37-c5fe6fbde415 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:07.841525 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-32ef8a70-9950-4b47-9a37-c5fe6fbde415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:07.850087 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1232/2458] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:39:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:39:07.854288 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7cc3c6-114e-4365-9f8f-6926a23af69b None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:07.864457 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7cc3c6-114e-4365-9f8f-6926a23af69b None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:07.869013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1f380-e063-4566-942e-d1c0fa86c9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:07.869607 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1f380-e063-4566-942e-d1c0fa86c9dc admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:07.869886 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1f380-e063-4566-942e-d1c0fa86c9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:07.870159 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1f380-e063-4566-942e-d1c0fa86c9dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:07.870831 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a1f380-e063-4566-942e-d1c0fa86c9dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:39:07.871136 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a1f380-e063-4566-942e-d1c0fa86c9dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:07.872742 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a1f380-e063-4566-942e-d1c0fa86c9dc admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:07.884714 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1227/2459] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:39:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:39:07.889296 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef8a70-9950-4b47-9a37-c5fe6fbde415 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:07.898468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef8a70-9950-4b47-9a37-c5fe6fbde415 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:07.903107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4bec7-61c7-40e3-a0d0-7eacdfd8edfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:07.903701 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4bec7-61c7-40e3-a0d0-7eacdfd8edfb admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:07.903984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4bec7-61c7-40e3-a0d0-7eacdfd8edfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:07.904260 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4bec7-61c7-40e3-a0d0-7eacdfd8edfb admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:07.904881 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c4bec7-61c7-40e3-a0d0-7eacdfd8edfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:07.906799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c4bec7-61c7-40e3-a0d0-7eacdfd8edfb admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:07.937038 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1233/2460] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:39:07 2022] POST /identity/v3/projects => generated 391 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:39:07.941215 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1f380-e063-4566-942e-d1c0fa86c9dc admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:07.951282 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1f380-e063-4566-942e-d1c0fa86c9dc admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:07.954912 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:08.010935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c609ba-45d7-4a04-a9fa-03f1216b8750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:08.011533 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c609ba-45d7-4a04-a9fa-03f1216b8750 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:08.011817 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c609ba-45d7-4a04-a9fa-03f1216b8750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:08.012091 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c609ba-45d7-4a04-a9fa-03f1216b8750 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:08.012724 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c609ba-45d7-4a04-a9fa-03f1216b8750 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:08.014721 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c609ba-45d7-4a04-a9fa-03f1216b8750 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:08.035864 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1228/2461] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:39:07 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:39:08.040131 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4bec7-61c7-40e3-a0d0-7eacdfd8edfb admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:08.050896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4bec7-61c7-40e3-a0d0-7eacdfd8edfb admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:08.055417 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a6b28-dd02-41fd-a4db-8c23c7670b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:08.056007 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a6b28-dd02-41fd-a4db-8c23c7670b5d admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:08.056285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a6b28-dd02-41fd-a4db-8c23c7670b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:08.056558 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a6b28-dd02-41fd-a4db-8c23c7670b5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:08.057190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a6b28-dd02-41fd-a4db-8c23c7670b5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:08.058800 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a6b28-dd02-41fd-a4db-8c23c7670b5d admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:08.069080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1234/2462] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:39:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:39:08.073027 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c609ba-45d7-4a04-a9fa-03f1216b8750 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:08.082913 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c609ba-45d7-4a04-a9fa-03f1216b8750 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:08.087441 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf4527-ea8f-4031-bee2-f5eaf557b02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:08.088028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf4527-ea8f-4031-bee2-f5eaf557b02a admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:08.088311 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf4527-ea8f-4031-bee2-f5eaf557b02a admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:08.088585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf4527-ea8f-4031-bee2-f5eaf557b02a admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:08.089220 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf4527-ea8f-4031-bee2-f5eaf557b02a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:08.090891 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf4527-ea8f-4031-bee2-f5eaf557b02a admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:08.102002 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1229/2463] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:39:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:39:08.106398 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a6b28-dd02-41fd-a4db-8c23c7670b5d admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:08.116935 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a6b28-dd02-41fd-a4db-8c23c7670b5d admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:08.121585 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c72b16e-cb40-42f2-9ed3-1baa71c1615e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:08.122243 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72b16e-cb40-42f2-9ed3-1baa71c1615e admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:08.122558 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72b16e-cb40-42f2-9ed3-1baa71c1615e admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:08.122836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72b16e-cb40-42f2-9ed3-1baa71c1615e admin admin] PATH_INFO: `/v3/projects/f21856f2a2b34d6391a80df87a0f7d40/users/01e282d5c3344005b8193e8bdb79b71b/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:08.137312 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c72b16e-cb40-42f2-9ed3-1baa71c1615e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f21856f2a2b34d6391a80df87a0f7d40, user_id=01e282d5c3344005b8193e8bdb79b71b, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:08.139997 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c72b16e-cb40-42f2-9ed3-1baa71c1615e admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:08.152723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1235/2464] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:39:08 2022] PUT /identity/v3/projects/f21856f2a2b34d6391a80df87a0f7d40/users/01e282d5c3344005b8193e8bdb79b71b/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:39:08.163235 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a085684-6cc8-4849-9024-55dd5f34c442 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:08.163528 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a085684-6cc8-4849-9024-55dd5f34c442 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:08.163807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a085684-6cc8-4849-9024-55dd5f34c442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:08.192002 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-2a085684-6cc8-4849-9024-55dd5f34c442 None None] MFA Rules not processed for user `01e282d5c3344005b8193e8bdb79b71b`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:08.193204 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-2a085684-6cc8-4849-9024-55dd5f34c442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:08.256225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1230/2465] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:39:08 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:39:08.267577 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c72b16e-cb40-42f2-9ed3-1baa71c1615e admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:08.269261 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-8c72b16e-cb40-42f2-9ed3-1baa71c1615e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:08.302402 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:08.331175 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c72b16e-cb40-42f2-9ed3-1baa71c1615e admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:08.334770 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3eae9-02a1-4ec5-9cd2-9e97eab56ade service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:08.335376 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3eae9-02a1-4ec5-9cd2-9e97eab56ade service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:08.335665 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3eae9-02a1-4ec5-9cd2-9e97eab56ade service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:08.335942 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3eae9-02a1-4ec5-9cd2-9e97eab56ade service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:08.344067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3eae9-02a1-4ec5-9cd2-9e97eab56ade service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:08.345516 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3eae9-02a1-4ec5-9cd2-9e97eab56ade service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:08.354654 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1236/2466] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:39:08 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:39:11.889844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a085684-6cc8-4849-9024-55dd5f34c442 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:11.899858 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a085684-6cc8-4849-9024-55dd5f34c442 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:11.903440 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec2168-32b3-4091-8749-b764c0cb651b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:11.904043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec2168-32b3-4091-8749-b764c0cb651b service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:11.904322 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec2168-32b3-4091-8749-b764c0cb651b service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:11.905082 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec2168-32b3-4091-8749-b764c0cb651b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:11.906854 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-cdec2168-32b3-4091-8749-b764c0cb651b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:11.972511 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec2168-32b3-4091-8749-b764c0cb651b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:11.974472 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec2168-32b3-4091-8749-b764c0cb651b service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:11.984350 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1231/2467] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:39:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:39:18.350609 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=14 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:21.979466 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=91,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:23.476063 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ca1e1-e01d-494d-aeea-a566367ed3ba None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:23.476558 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ca1e1-e01d-494d-aeea-a566367ed3ba None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:23.476756 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ca1e1-e01d-494d-aeea-a566367ed3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:23.495902 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-153ca1e1-e01d-494d-aeea-a566367ed3ba None None] MFA Rules not processed for user `73624e79ccdd4782beddf663bb6d02e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:23.497099 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-153ca1e1-e01d-494d-aeea-a566367ed3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:23.504806 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1237/2468] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:39:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:39:23.830455 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec2168-32b3-4091-8749-b764c0cb651b service cinder] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:23.831850 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-cdec2168-32b3-4091-8749-b764c0cb651b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:23.895241 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec2168-32b3-4091-8749-b764c0cb651b service cinder] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:23.898795 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c88d205-a09d-4f78-b0e9-b933e92b0698 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:23.899397 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c88d205-a09d-4f78-b0e9-b933e92b0698 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:23.899678 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c88d205-a09d-4f78-b0e9-b933e92b0698 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:23.899948 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c88d205-a09d-4f78-b0e9-b933e92b0698 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:23.901639 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-8c88d205-a09d-4f78-b0e9-b933e92b0698 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:23.911595 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c88d205-a09d-4f78-b0e9-b933e92b0698 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:23.913028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c88d205-a09d-4f78-b0e9-b933e92b0698 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:23.921393 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1232/2469] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:39:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:39:24.231558 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ca1e1-e01d-494d-aeea-a566367ed3ba None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:24.233025 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-153ca1e1-e01d-494d-aeea-a566367ed3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:24.303544 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ca1e1-e01d-494d-aeea-a566367ed3ba None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:24.308477 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38d4e9-33e5-42b9-9dfc-783e3b2a6e2c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ddfb151bbf844efae73ec1990e7eaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:24.309092 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38d4e9-33e5-42b9-9dfc-783e3b2a6e2c service placement] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:24.309375 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38d4e9-33e5-42b9-9dfc-783e3b2a6e2c service placement] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:24.309772 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38d4e9-33e5-42b9-9dfc-783e3b2a6e2c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:24.311498 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7b38d4e9-33e5-42b9-9dfc-783e3b2a6e2c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:24.322991 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b38d4e9-33e5-42b9-9dfc-783e3b2a6e2c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:24.324559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b38d4e9-33e5-42b9-9dfc-783e3b2a6e2c service placement] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:24.334628 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1238/2470] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:39:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:39:26.452272 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c88d205-a09d-4f78-b0e9-b933e92b0698 service neutron] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:26.465941 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c88d205-a09d-4f78-b0e9-b933e92b0698 service neutron] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:26.470674 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbad06b-64e9-403c-8669-587fc60dd9cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:26.471303 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbad06b-64e9-403c-8669-587fc60dd9cd service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:26.471585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbad06b-64e9-403c-8669-587fc60dd9cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:26.471866 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbad06b-64e9-403c-8669-587fc60dd9cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:26.474273 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-abbad06b-64e9-403c-8669-587fc60dd9cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:26.485667 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbad06b-64e9-403c-8669-587fc60dd9cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:26.487203 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbad06b-64e9-403c-8669-587fc60dd9cd service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:26.496069 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1233/2471] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:39:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:39:26.654210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38d4e9-33e5-42b9-9dfc-783e3b2a6e2c service placement] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:26.719238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38d4e9-33e5-42b9-9dfc-783e3b2a6e2c service placement] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:26.724016 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b6c45-5e1a-4f5a-bd23-5774f059fbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:26.724643 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b6c45-5e1a-4f5a-bd23-5774f059fbe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:26.724924 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b6c45-5e1a-4f5a-bd23-5774f059fbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:26.725198 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b6c45-5e1a-4f5a-bd23-5774f059fbe5 admin admin] PATH_INFO: `/v3/users/01e282d5c3344005b8193e8bdb79b71b` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:26.726365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b6c45-5e1a-4f5a-bd23-5774f059fbe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01e282d5c3344005b8193e8bdb79b71b)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:26.728270 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b6c45-5e1a-4f5a-bd23-5774f059fbe5 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:26.776650 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3b1b6c45-5e1a-4f5a-bd23-5774f059fbe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01e282d5c3344005b8193e8bdb79b71b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:26.784524 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3b1b6c45-5e1a-4f5a-bd23-5774f059fbe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01e282d5c3344005b8193e8bdb79b71b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:26.785267 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3b1b6c45-5e1a-4f5a-bd23-5774f059fbe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01e282d5c3344005b8193e8bdb79b71b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:26.798574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3b1b6c45-5e1a-4f5a-bd23-5774f059fbe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01e282d5c3344005b8193e8bdb79b71b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:26.799237 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3b1b6c45-5e1a-4f5a-bd23-5774f059fbe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01e282d5c3344005b8193e8bdb79b71b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:26.811936 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1239/2472] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:39:26 2022] DELETE /identity/v3/users/01e282d5c3344005b8193e8bdb79b71b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:39:26.816440 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959ef3a-3ea2-45a6-b43b-033ebfdf294c None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:26.816916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959ef3a-3ea2-45a6-b43b-033ebfdf294c None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:26.817181 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959ef3a-3ea2-45a6-b43b-033ebfdf294c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:26.836013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-f959ef3a-3ea2-45a6-b43b-033ebfdf294c None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:26.837203 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-f959ef3a-3ea2-45a6-b43b-033ebfdf294c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:26.899499 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1234/2473] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:39:26.910183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b6c45-5e1a-4f5a-bd23-5774f059fbe5 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:26.911729 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3b1b6c45-5e1a-4f5a-bd23-5774f059fbe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:26.917410 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:26.983453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b6c45-5e1a-4f5a-bd23-5774f059fbe5 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:26.987431 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f7e3f-e689-4cd2-9578-184c5815bce7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:26.988252 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f7e3f-e689-4cd2-9578-184c5815bce7 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:26.988536 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f7e3f-e689-4cd2-9578-184c5815bce7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:26.988802 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f7e3f-e689-4cd2-9578-184c5815bce7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:26.999302 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f7e3f-e689-4cd2-9578-184c5815bce7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:27.001010 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f7e3f-e689-4cd2-9578-184c5815bce7 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:27.015038 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1240/2474] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:39:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:39:27.103023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f959ef3a-3ea2-45a6-b43b-033ebfdf294c None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:27.106311 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-f959ef3a-3ea2-45a6-b43b-033ebfdf294c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:27.216397 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f959ef3a-3ea2-45a6-b43b-033ebfdf294c None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:27.224777 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3bb4ff-f91a-47eb-841f-89a1a8878756 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:27.225712 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bb4ff-f91a-47eb-841f-89a1a8878756 service nova] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:27.225712 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bb4ff-f91a-47eb-841f-89a1a8878756 service nova] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:27.225951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bb4ff-f91a-47eb-841f-89a1a8878756 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:27.227952 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-7c3bb4ff-f91a-47eb-841f-89a1a8878756 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:27.240990 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3bb4ff-f91a-47eb-841f-89a1a8878756 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:27.242785 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3bb4ff-f91a-47eb-841f-89a1a8878756 service nova] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:27.254075 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1235/2475] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:39:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:39:27.292707 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f7e3f-e689-4cd2-9578-184c5815bce7 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:27.294228 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7c2f7e3f-e689-4cd2-9578-184c5815bce7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:27.309615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f7e3f-e689-4cd2-9578-184c5815bce7 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:27.315155 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad531eb0-ecd6-47b7-9fbc-525b5bd2518b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:27.315822 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad531eb0-ecd6-47b7-9fbc-525b5bd2518b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:27.316109 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad531eb0-ecd6-47b7-9fbc-525b5bd2518b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:27.316385 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad531eb0-ecd6-47b7-9fbc-525b5bd2518b admin admin] PATH_INFO: `/v3/projects/f21856f2a2b34d6391a80df87a0f7d40` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:27.317524 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad531eb0-ecd6-47b7-9fbc-525b5bd2518b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f21856f2a2b34d6391a80df87a0f7d40)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:27.319512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad531eb0-ecd6-47b7-9fbc-525b5bd2518b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:27.378779 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-ad531eb0-ecd6-47b7-9fbc-525b5bd2518b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f21856f2a2b34d6391a80df87a0f7d40', 'request_id': 'req-ad531eb0-ecd6-47b7-9fbc-525b5bd2518b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:27.385202 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-ad531eb0-ecd6-47b7-9fbc-525b5bd2518b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f21856f2a2b34d6391a80df87a0f7d40', 'request_id': 'req-ad531eb0-ecd6-47b7-9fbc-525b5bd2518b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:27.393414 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-ad531eb0-ecd6-47b7-9fbc-525b5bd2518b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f21856f2a2b34d6391a80df87a0f7d40', 'request_id': 'req-ad531eb0-ecd6-47b7-9fbc-525b5bd2518b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:27.405535 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-ad531eb0-ecd6-47b7-9fbc-525b5bd2518b admin admin] The token cache is being invalidate because project f21856f2a2b34d6391a80df87a0f7d40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:39:27.405760 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-ad531eb0-ecd6-47b7-9fbc-525b5bd2518b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:27.406467 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-ad531eb0-ecd6-47b7-9fbc-525b5bd2518b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:27.407965 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1241/2476] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:39:27 2022] DELETE /identity/v3/projects/f21856f2a2b34d6391a80df87a0f7d40 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:39:27.413405 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35edb9-d252-45eb-bce9-21bb6c9e7ed5 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:27.413735 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35edb9-d252-45eb-bce9-21bb6c9e7ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:27.414004 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35edb9-d252-45eb-bce9-21bb6c9e7ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:27.433481 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-ca35edb9-d252-45eb-bce9-21bb6c9e7ed5 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:27.434719 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-ca35edb9-d252-45eb-bce9-21bb6c9e7ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:27.444909 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:27.497265 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1236/2477] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:39:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:39:27.505827 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af1145-2d79-4fdf-bf75-4b127a2822ad None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:27.506116 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af1145-2d79-4fdf-bf75-4b127a2822ad None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:27.506468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af1145-2d79-4fdf-bf75-4b127a2822ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:27.526931 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-f6af1145-2d79-4fdf-bf75-4b127a2822ad None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:27.528153 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f6af1145-2d79-4fdf-bf75-4b127a2822ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:27.536194 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1242/2478] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:39:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:39:27.540512 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35edb9-d252-45eb-bce9-21bb6c9e7ed5 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:27.551085 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35edb9-d252-45eb-bce9-21bb6c9e7ed5 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:27.555710 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de688e52-498b-4048-8012-0b719f8df3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:27.556329 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de688e52-498b-4048-8012-0b719f8df3be admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:27.556589 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de688e52-498b-4048-8012-0b719f8df3be admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:27.556889 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de688e52-498b-4048-8012-0b719f8df3be admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:27.557891 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de688e52-498b-4048-8012-0b719f8df3be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:39:27.557891 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de688e52-498b-4048-8012-0b719f8df3be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:27.559482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de688e52-498b-4048-8012-0b719f8df3be admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:27.571856 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1237/2479] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:39:27.576810 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af1145-2d79-4fdf-bf75-4b127a2822ad None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:27.586009 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af1145-2d79-4fdf-bf75-4b127a2822ad None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:27.591810 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f7778-3a6b-4caa-b118-93fd2007b897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:27.592423 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f7778-3a6b-4caa-b118-93fd2007b897 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:27.592704 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f7778-3a6b-4caa-b118-93fd2007b897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:27.592981 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f7778-3a6b-4caa-b118-93fd2007b897 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:27.593613 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f7778-3a6b-4caa-b118-93fd2007b897 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:27.595539 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f7778-3a6b-4caa-b118-93fd2007b897 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:27.630503 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1243/2480] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:39:27 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:39:27.636187 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de688e52-498b-4048-8012-0b719f8df3be admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:27.647376 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de688e52-498b-4048-8012-0b719f8df3be admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:27.706796 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc99200-a54c-4667-95bf-c999bbf403db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:27.707411 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc99200-a54c-4667-95bf-c999bbf403db admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:27.707692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc99200-a54c-4667-95bf-c999bbf403db admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:27.707968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc99200-a54c-4667-95bf-c999bbf403db admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:27.708610 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc99200-a54c-4667-95bf-c999bbf403db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:27.710631 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc99200-a54c-4667-95bf-c999bbf403db admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:27.732107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1238/2481] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:39:27 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:39:27.736502 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f7778-3a6b-4caa-b118-93fd2007b897 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:27.746615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f7778-3a6b-4caa-b118-93fd2007b897 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:27.752055 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753d7a0-6b8c-4337-8863-c8258addb600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:27.752659 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753d7a0-6b8c-4337-8863-c8258addb600 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:27.752944 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753d7a0-6b8c-4337-8863-c8258addb600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:27.753220 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753d7a0-6b8c-4337-8863-c8258addb600 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:27.753868 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753d7a0-6b8c-4337-8863-c8258addb600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:27.755482 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753d7a0-6b8c-4337-8863-c8258addb600 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:27.766002 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1244/2482] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:39:27.769892 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc99200-a54c-4667-95bf-c999bbf403db admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:27.780034 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc99200-a54c-4667-95bf-c999bbf403db admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:27.784590 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2cd18a-6fbe-409a-bc38-00cc51b4b361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:27.785183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cd18a-6fbe-409a-bc38-00cc51b4b361 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:27.785471 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cd18a-6fbe-409a-bc38-00cc51b4b361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:27.785746 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cd18a-6fbe-409a-bc38-00cc51b4b361 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:27.786427 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2cd18a-6fbe-409a-bc38-00cc51b4b361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:27.788018 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2cd18a-6fbe-409a-bc38-00cc51b4b361 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:27.798404 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1239/2483] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:39:27.802255 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753d7a0-6b8c-4337-8863-c8258addb600 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:27.812473 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753d7a0-6b8c-4337-8863-c8258addb600 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:27.817073 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d930e-cc4f-40c3-a83f-35ee87371f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:27.817833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d930e-cc4f-40c3-a83f-35ee87371f09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:27.818025 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d930e-cc4f-40c3-a83f-35ee87371f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:27.818314 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d930e-cc4f-40c3-a83f-35ee87371f09 admin admin] PATH_INFO: `/v3/projects/b683a11c21fe4dd389a006f051576a1b/users/aa6ad9c2c1294633a74782c668227886/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:27.834359 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d930e-cc4f-40c3-a83f-35ee87371f09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b683a11c21fe4dd389a006f051576a1b, user_id=aa6ad9c2c1294633a74782c668227886, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:27.836982 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d930e-cc4f-40c3-a83f-35ee87371f09 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:27.849922 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1245/2484] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:39:27 2022] PUT /identity/v3/projects/b683a11c21fe4dd389a006f051576a1b/users/aa6ad9c2c1294633a74782c668227886/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:39:27.855055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e18895-f9ea-4b3a-bd15-4371a135716b None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:27.855358 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e18895-f9ea-4b3a-bd15-4371a135716b None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:27.855683 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e18895-f9ea-4b3a-bd15-4371a135716b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:27.875304 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-87e18895-f9ea-4b3a-bd15-4371a135716b None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:27.876509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-87e18895-f9ea-4b3a-bd15-4371a135716b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:27.933663 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1240/2485] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:39:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:39:27.943897 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d930e-cc4f-40c3-a83f-35ee87371f09 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:27.945590 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-b19d930e-cc4f-40c3-a83f-35ee87371f09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:27.972800 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:28.003607 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d930e-cc4f-40c3-a83f-35ee87371f09 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:28.007246 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23494800-af48-450f-a8a4-c05cdbd021e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:28.007851 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23494800-af48-450f-a8a4-c05cdbd021e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:28.008135 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23494800-af48-450f-a8a4-c05cdbd021e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:28.008409 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23494800-af48-450f-a8a4-c05cdbd021e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:28.017170 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23494800-af48-450f-a8a4-c05cdbd021e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:28.018656 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23494800-af48-450f-a8a4-c05cdbd021e6 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:28.027828 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1246/2486] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:39:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:39:32.260907 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdc2e5-e990-4088-81c6-665c0175d8cd None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:32.261446 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdc2e5-e990-4088-81c6-665c0175d8cd None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:32.261446 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdc2e5-e990-4088-81c6-665c0175d8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:32.291724 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-a8cdc2e5-e990-4088-81c6-665c0175d8cd None None] MFA Rules not processed for user `aa6ad9c2c1294633a74782c668227886`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:32.293074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-a8cdc2e5-e990-4088-81c6-665c0175d8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:32.303779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:32.375081 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1241/2487] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:39:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:39:32.379319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23494800-af48-450f-a8a4-c05cdbd021e6 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:32.390488 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23494800-af48-450f-a8a4-c05cdbd021e6 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:32.395291 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57bc8c7-919f-44f7-a3a5-3d7796db1aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:32.396123 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57bc8c7-919f-44f7-a3a5-3d7796db1aa9 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:32.396581 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57bc8c7-919f-44f7-a3a5-3d7796db1aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:32.397008 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57bc8c7-919f-44f7-a3a5-3d7796db1aa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:32.397808 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57bc8c7-919f-44f7-a3a5-3d7796db1aa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:32.399924 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57bc8c7-919f-44f7-a3a5-3d7796db1aa9 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:32.439731 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1247/2488] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:39:32 2022] POST /identity/v3/projects => generated 369 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:39:32.443921 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cdc2e5-e990-4088-81c6-665c0175d8cd None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:32.455666 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cdc2e5-e990-4088-81c6-665c0175d8cd None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:32.530343 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65318aa1-86fb-40b1-a8f0-12ed75bcef0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:32.531142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65318aa1-86fb-40b1-a8f0-12ed75bcef0a admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:32.531252 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65318aa1-86fb-40b1-a8f0-12ed75bcef0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:32.531499 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65318aa1-86fb-40b1-a8f0-12ed75bcef0a admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:32.532132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65318aa1-86fb-40b1-a8f0-12ed75bcef0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:32.534711 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65318aa1-86fb-40b1-a8f0-12ed75bcef0a admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:32.560389 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1242/2489] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:39:32 2022] POST /identity/v3/users => generated 291 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:39:32.565110 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57bc8c7-919f-44f7-a3a5-3d7796db1aa9 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:32.578773 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57bc8c7-919f-44f7-a3a5-3d7796db1aa9 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:32.585764 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08cb03-96c2-451b-a8d7-70b6bda85e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:32.586740 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08cb03-96c2-451b-a8d7-70b6bda85e3f admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:32.587109 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08cb03-96c2-451b-a8d7-70b6bda85e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:32.587484 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08cb03-96c2-451b-a8d7-70b6bda85e3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:32.588355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08cb03-96c2-451b-a8d7-70b6bda85e3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:32.590100 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08cb03-96c2-451b-a8d7-70b6bda85e3f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:32.604056 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1248/2490] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:39:32.607965 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65318aa1-86fb-40b1-a8f0-12ed75bcef0a admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:32.619713 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65318aa1-86fb-40b1-a8f0-12ed75bcef0a admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:32.625463 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6007af2-d380-4ccc-96f1-3eef8bd613e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:32.626093 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6007af2-d380-4ccc-96f1-3eef8bd613e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:32.626397 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6007af2-d380-4ccc-96f1-3eef8bd613e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:32.626669 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6007af2-d380-4ccc-96f1-3eef8bd613e6 admin admin] PATH_INFO: `/v3/projects/5b4f3cdd63e64925afa07955c7aaf518/users/a477a6b7a6724c40a0efb808edfef2a4/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:32.643363 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6007af2-d380-4ccc-96f1-3eef8bd613e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b4f3cdd63e64925afa07955c7aaf518, user_id=a477a6b7a6724c40a0efb808edfef2a4, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:32.646379 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6007af2-d380-4ccc-96f1-3eef8bd613e6 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:32.661508 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1243/2491] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:39:32 2022] PUT /identity/v3/projects/5b4f3cdd63e64925afa07955c7aaf518/users/a477a6b7a6724c40a0efb808edfef2a4/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:39:32.665372 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08cb03-96c2-451b-a8d7-70b6bda85e3f admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:32.688283 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08cb03-96c2-451b-a8d7-70b6bda85e3f admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:32.716058 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6007af2-d380-4ccc-96f1-3eef8bd613e6 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:32.718074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-e6007af2-d380-4ccc-96f1-3eef8bd613e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:32.801686 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d00e45-feb6-4cc1-bae0-7b2e88639555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:32.801946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6007af2-d380-4ccc-96f1-3eef8bd613e6 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:32.802909 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d00e45-feb6-4cc1-bae0-7b2e88639555 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:32.803093 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d00e45-feb6-4cc1-bae0-7b2e88639555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:32.803321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d00e45-feb6-4cc1-bae0-7b2e88639555 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:32.804078 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d00e45-feb6-4cc1-bae0-7b2e88639555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:32.805778 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d00e45-feb6-4cc1-bae0-7b2e88639555 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:32.806930 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7cd98e-5f54-4682-a37c-b593a235d9b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:32.808122 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7cd98e-5f54-4682-a37c-b593a235d9b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:32.808344 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7cd98e-5f54-4682-a37c-b593a235d9b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:32.808636 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7cd98e-5f54-4682-a37c-b593a235d9b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:32.811268 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-0d7cd98e-5f54-4682-a37c-b593a235d9b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:32.819574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1249/2492] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:39:32.824607 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d00e45-feb6-4cc1-bae0-7b2e88639555 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:32.838845 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d00e45-feb6-4cc1-bae0-7b2e88639555 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:32.844829 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6900fe-6d17-4463-be80-5090ef6d9f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:32.845857 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6900fe-6d17-4463-be80-5090ef6d9f46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:32.846143 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6900fe-6d17-4463-be80-5090ef6d9f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:32.846461 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6900fe-6d17-4463-be80-5090ef6d9f46 admin admin] PATH_INFO: `/v3/projects/5b4f3cdd63e64925afa07955c7aaf518/users/a477a6b7a6724c40a0efb808edfef2a4/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:32.848898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6900fe-6d17-4463-be80-5090ef6d9f46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b4f3cdd63e64925afa07955c7aaf518, user_id=a477a6b7a6724c40a0efb808edfef2a4, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:32.852074 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6900fe-6d17-4463-be80-5090ef6d9f46 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:32.862214 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:32.866685 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1250/2493] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:39:32 2022] PUT /identity/v3/projects/5b4f3cdd63e64925afa07955c7aaf518/users/a477a6b7a6724c40a0efb808edfef2a4/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:39:32.913115 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7cd98e-5f54-4682-a37c-b593a235d9b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:32.914766 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7cd98e-5f54-4682-a37c-b593a235d9b2 service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:32.930830 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1244/2494] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:39:32 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:39:38.377626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e780046-f27d-4cb3-b06b-8d0207d6e420 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:38.378192 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e780046-f27d-4cb3-b06b-8d0207d6e420 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:38.379914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e780046-f27d-4cb3-b06b-8d0207d6e420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:38.423295 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-7e780046-f27d-4cb3-b06b-8d0207d6e420 None None] MFA Rules not processed for user `a477a6b7a6724c40a0efb808edfef2a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:38.424793 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7e780046-f27d-4cb3-b06b-8d0207d6e420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:38.514410 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1251/2495] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:39:38 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:39:38.526960 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7cd98e-5f54-4682-a37c-b593a235d9b2 service cinder] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:38.528537 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-0d7cd98e-5f54-4682-a37c-b593a235d9b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:38.637686 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7cd98e-5f54-4682-a37c-b593a235d9b2 service cinder] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:38.732586 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6460ec-2a2c-43c9-a691-a45a88bfe241 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:38.732866 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6460ec-2a2c-43c9-a691-a45a88bfe241 service nova] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:38.733065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6460ec-2a2c-43c9-a691-a45a88bfe241 service nova] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:38.733341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6460ec-2a2c-43c9-a691-a45a88bfe241 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:38.818362 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6460ec-2a2c-43c9-a691-a45a88bfe241 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:38.820191 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6460ec-2a2c-43c9-a691-a45a88bfe241 service nova] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:38.835235 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1245/2496] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:39:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:39:39.932332 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dbeeb-dca0-421b-9422-905de4c4fb78 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:39.932775 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dbeeb-dca0-421b-9422-905de4c4fb78 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:39.934912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dbeeb-dca0-421b-9422-905de4c4fb78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:39.941434 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:39.957803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-d47dbeeb-dca0-421b-9422-905de4c4fb78 None None] MFA Rules not processed for user `aa6ad9c2c1294633a74782c668227886`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:39.963638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d47dbeeb-dca0-421b-9422-905de4c4fb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:39.974088 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1252/2497] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:39:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:39:48.522593 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6460ec-2a2c-43c9-a691-a45a88bfe241 service nova] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:48.534580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6460ec-2a2c-43c9-a691-a45a88bfe241 service nova] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:48.540457 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4941368-1028-4b0a-8313-3a532ffb2f19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:48.541088 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4941368-1028-4b0a-8313-3a532ffb2f19 service nova] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:48.541372 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4941368-1028-4b0a-8313-3a532ffb2f19 service nova] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:48.541646 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4941368-1028-4b0a-8313-3a532ffb2f19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:48.543393 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-c4941368-1028-4b0a-8313-3a532ffb2f19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:48.549593 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:48.607341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4941368-1028-4b0a-8313-3a532ffb2f19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:48.608957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4941368-1028-4b0a-8313-3a532ffb2f19 service nova] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:48.618044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1246/2498] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:39:48 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:39:49.951420 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:57.976528 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47dbeeb-dca0-421b-9422-905de4c4fb78 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:58.082742 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47dbeeb-dca0-421b-9422-905de4c4fb78 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:58.086421 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac701e60-2578-421b-98b8-9b6ba55d66b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:58.087169 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac701e60-2578-421b-98b8-9b6ba55d66b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:58.087557 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac701e60-2578-421b-98b8-9b6ba55d66b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:58.087923 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac701e60-2578-421b-98b8-9b6ba55d66b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:58.089888 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ac701e60-2578-421b-98b8-9b6ba55d66b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:58.105016 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac701e60-2578-421b-98b8-9b6ba55d66b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:58.106818 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac701e60-2578-421b-98b8-9b6ba55d66b6 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:58.116653 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1253/2499] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:39:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:39:58.614016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=25 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:59.144401 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4941368-1028-4b0a-8313-3a532ffb2f19 service nova] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:59.146379 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-c4941368-1028-4b0a-8313-3a532ffb2f19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:59.168080 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4941368-1028-4b0a-8313-3a532ffb2f19 service nova] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:59.175289 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc1005b-a8c5-4f3d-9ec4-1991758cb5f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:59.176150 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1005b-a8c5-4f3d-9ec4-1991758cb5f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:59.176878 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1005b-a8c5-4f3d-9ec4-1991758cb5f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:59.177243 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1005b-a8c5-4f3d-9ec4-1991758cb5f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:59.179325 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-dcc1005b-a8c5-4f3d-9ec4-1991758cb5f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:59.257763 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc1005b-a8c5-4f3d-9ec4-1991758cb5f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:59.259606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc1005b-a8c5-4f3d-9ec4-1991758cb5f2 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:59.270572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1247/2500] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:39:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:39:59.618952 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57597f88-6416-40d6-92d8-3267764b76ec None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:59.619499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57597f88-6416-40d6-92d8-3267764b76ec None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:59.619711 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57597f88-6416-40d6-92d8-3267764b76ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:59.643556 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-57597f88-6416-40d6-92d8-3267764b76ec None None] MFA Rules not processed for user `a3059f5115bb4bad837daf9f18752143`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:59.644955 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-57597f88-6416-40d6-92d8-3267764b76ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:59.708825 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1254/2501] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:39:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:40:06.590740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc1005b-a8c5-4f3d-9ec4-1991758cb5f2 service neutron] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:06.593241 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-dcc1005b-a8c5-4f3d-9ec4-1991758cb5f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:06.608264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc1005b-a8c5-4f3d-9ec4-1991758cb5f2 service neutron] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:06.613075 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9a234-be5c-4dff-97dc-ba2783a37fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:06.613630 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9a234-be5c-4dff-97dc-ba2783a37fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:06.613931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9a234-be5c-4dff-97dc-ba2783a37fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:06.619159 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9a234-be5c-4dff-97dc-ba2783a37fd9 admin admin] PATH_INFO: `/v3/users/8020db56bca04d61b5a6e40384c25715` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:06.635840 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b9a234-be5c-4dff-97dc-ba2783a37fd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8020db56bca04d61b5a6e40384c25715)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:06.638138 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b9a234-be5c-4dff-97dc-ba2783a37fd9 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:06.704397 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-46b9a234-be5c-4dff-97dc-ba2783a37fd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8020db56bca04d61b5a6e40384c25715'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:06.705061 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-46b9a234-be5c-4dff-97dc-ba2783a37fd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8020db56bca04d61b5a6e40384c25715'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:06.715527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-46b9a234-be5c-4dff-97dc-ba2783a37fd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8020db56bca04d61b5a6e40384c25715'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:06.716198 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-46b9a234-be5c-4dff-97dc-ba2783a37fd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8020db56bca04d61b5a6e40384c25715'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:06.739040 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-46b9a234-be5c-4dff-97dc-ba2783a37fd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8020db56bca04d61b5a6e40384c25715'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:06.754843 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1248/2502] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:40:06 2022] DELETE /identity/v3/users/8020db56bca04d61b5a6e40384c25715 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:06.979701 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57597f88-6416-40d6-92d8-3267764b76ec None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:06.992174 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57597f88-6416-40d6-92d8-3267764b76ec None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:07.052754 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e75c900-60dc-44c2-9512-5354ecf4979d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:07.053502 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75c900-60dc-44c2-9512-5354ecf4979d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:07.053784 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75c900-60dc-44c2-9512-5354ecf4979d admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:07.054060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75c900-60dc-44c2-9512-5354ecf4979d admin admin] PATH_INFO: `/v3/projects/b4bbb5d3895c49fa8695d3bca946dbea` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:07.055234 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e75c900-60dc-44c2-9512-5354ecf4979d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4bbb5d3895c49fa8695d3bca946dbea)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:07.057139 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e75c900-60dc-44c2-9512-5354ecf4979d admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:07.113777 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9e75c900-60dc-44c2-9512-5354ecf4979d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4bbb5d3895c49fa8695d3bca946dbea', 'request_id': 'req-9e75c900-60dc-44c2-9512-5354ecf4979d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:07.116668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:07.120988 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9e75c900-60dc-44c2-9512-5354ecf4979d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4bbb5d3895c49fa8695d3bca946dbea', 'request_id': 'req-9e75c900-60dc-44c2-9512-5354ecf4979d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:07.130600 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9e75c900-60dc-44c2-9512-5354ecf4979d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4bbb5d3895c49fa8695d3bca946dbea', 'request_id': 'req-9e75c900-60dc-44c2-9512-5354ecf4979d'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:07.150547 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9e75c900-60dc-44c2-9512-5354ecf4979d admin admin] The token cache is being invalidate because project b4bbb5d3895c49fa8695d3bca946dbea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:40:07.151184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9e75c900-60dc-44c2-9512-5354ecf4979d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:07.151952 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-9e75c900-60dc-44c2-9512-5354ecf4979d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:07.154769 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1255/2503] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:40:06 2022] DELETE /identity/v3/projects/b4bbb5d3895c49fa8695d3bca946dbea => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:07.159383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9791a56-871d-48f3-9a4f-f4f7d8425994 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:07.159676 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9791a56-871d-48f3-9a4f-f4f7d8425994 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:07.159958 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9791a56-871d-48f3-9a4f-f4f7d8425994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:07.181748 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-a9791a56-871d-48f3-9a4f-f4f7d8425994 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:07.182999 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-a9791a56-871d-48f3-9a4f-f4f7d8425994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:07.247572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1249/2504] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:07.254823 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03285930-acfb-4ddf-9c19-4bc1a9f24c5d None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:07.255120 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03285930-acfb-4ddf-9c19-4bc1a9f24c5d None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:07.255405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03285930-acfb-4ddf-9c19-4bc1a9f24c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:07.276406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-03285930-acfb-4ddf-9c19-4bc1a9f24c5d None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:07.277658 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-03285930-acfb-4ddf-9c19-4bc1a9f24c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:07.285579 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1256/2505] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:40:07.289954 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9791a56-871d-48f3-9a4f-f4f7d8425994 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:07.299658 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9791a56-871d-48f3-9a4f-f4f7d8425994 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:07.304737 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f45a0-eb2f-4755-b8e1-5d073b3f68f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:07.305416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f45a0-eb2f-4755-b8e1-5d073b3f68f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:07.306068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f45a0-eb2f-4755-b8e1-5d073b3f68f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:07.306068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f45a0-eb2f-4755-b8e1-5d073b3f68f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:07.306618 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77f45a0-eb2f-4755-b8e1-5d073b3f68f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:40:07.306917 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77f45a0-eb2f-4755-b8e1-5d073b3f68f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:07.308542 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77f45a0-eb2f-4755-b8e1-5d073b3f68f8 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:07.310679 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:07.324326 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1250/2506] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:40:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:40:07.329408 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03285930-acfb-4ddf-9c19-4bc1a9f24c5d None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:07.339073 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03285930-acfb-4ddf-9c19-4bc1a9f24c5d None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:07.344004 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a6050-404e-40cf-b6ff-163b6dbe24a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:07.344739 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a6050-404e-40cf-b6ff-163b6dbe24a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:07.345094 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a6050-404e-40cf-b6ff-163b6dbe24a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:07.345431 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a6050-404e-40cf-b6ff-163b6dbe24a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:07.346141 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a6050-404e-40cf-b6ff-163b6dbe24a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:07.348467 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a6050-404e-40cf-b6ff-163b6dbe24a0 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:07.376561 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1257/2507] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:40:07 2022] POST /identity/v3/projects => generated 405 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:40:07.380684 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f45a0-eb2f-4755-b8e1-5d073b3f68f8 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:07.391033 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f45a0-eb2f-4755-b8e1-5d073b3f68f8 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:07.445486 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ead2d2-79c2-489e-81bd-1fd327730eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:07.446077 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ead2d2-79c2-489e-81bd-1fd327730eac admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:07.446388 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ead2d2-79c2-489e-81bd-1fd327730eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:07.446687 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ead2d2-79c2-489e-81bd-1fd327730eac admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:07.447298 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ead2d2-79c2-489e-81bd-1fd327730eac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:07.449236 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ead2d2-79c2-489e-81bd-1fd327730eac admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:07.470388 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1251/2508] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:40:07 2022] POST /identity/v3/users => generated 303 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:40:07.474601 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a6050-404e-40cf-b6ff-163b6dbe24a0 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:07.484951 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a6050-404e-40cf-b6ff-163b6dbe24a0 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:07.489632 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a248eca0-52a1-4974-ae0e-8051edf53366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:07.490293 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248eca0-52a1-4974-ae0e-8051edf53366 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:07.490616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248eca0-52a1-4974-ae0e-8051edf53366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:07.490895 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248eca0-52a1-4974-ae0e-8051edf53366 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:07.491540 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a248eca0-52a1-4974-ae0e-8051edf53366 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:07.493191 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a248eca0-52a1-4974-ae0e-8051edf53366 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:07.504133 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1258/2509] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:40:07.507845 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ead2d2-79c2-489e-81bd-1fd327730eac admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:07.518143 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ead2d2-79c2-489e-81bd-1fd327730eac admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:07.522676 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a708d-1612-4e0e-a666-2749421820a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:07.523258 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a708d-1612-4e0e-a666-2749421820a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:07.523536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a708d-1612-4e0e-a666-2749421820a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:07.523807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a708d-1612-4e0e-a666-2749421820a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:07.524438 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a708d-1612-4e0e-a666-2749421820a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:07.526020 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a708d-1612-4e0e-a666-2749421820a2 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:07.536418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1252/2510] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:40:07.540317 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a248eca0-52a1-4974-ae0e-8051edf53366 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:07.550355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a248eca0-52a1-4974-ae0e-8051edf53366 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:07.554830 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f8632-ff9e-40a2-a201-76aa2c08c22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:07.555464 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f8632-ff9e-40a2-a201-76aa2c08c22a admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:07.555745 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f8632-ff9e-40a2-a201-76aa2c08c22a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:07.556020 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f8632-ff9e-40a2-a201-76aa2c08c22a admin admin] PATH_INFO: `/v3/projects/ec6cd5c5d62643c2898f6d7188f66252/users/2fd04fb6525f4a52b8584d934a0be9ab/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:07.570793 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2f8632-ff9e-40a2-a201-76aa2c08c22a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec6cd5c5d62643c2898f6d7188f66252, user_id=2fd04fb6525f4a52b8584d934a0be9ab, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:07.573500 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2f8632-ff9e-40a2-a201-76aa2c08c22a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:07.586740 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1259/2511] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:40:07 2022] PUT /identity/v3/projects/ec6cd5c5d62643c2898f6d7188f66252/users/2fd04fb6525f4a52b8584d934a0be9ab/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:07.598170 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479eca5-3b42-45d7-9936-d73aa01d289c None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:07.598694 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479eca5-3b42-45d7-9936-d73aa01d289c None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:07.598789 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479eca5-3b42-45d7-9936-d73aa01d289c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:07.627557 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-1479eca5-3b42-45d7-9936-d73aa01d289c None None] MFA Rules not processed for user `2fd04fb6525f4a52b8584d934a0be9ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:07.628764 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-1479eca5-3b42-45d7-9936-d73aa01d289c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:07.695371 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1253/2512] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:40:07 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:07.699884 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f8632-ff9e-40a2-a201-76aa2c08c22a admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:07.764231 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f8632-ff9e-40a2-a201-76aa2c08c22a admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:07.768992 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b5079-ca62-48a3-9888-3505f9c9ff5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:07.769596 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b5079-ca62-48a3-9888-3505f9c9ff5b admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:07.769902 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b5079-ca62-48a3-9888-3505f9c9ff5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:07.770176 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b5079-ca62-48a3-9888-3505f9c9ff5b admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:07.770949 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b5079-ca62-48a3-9888-3505f9c9ff5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:07.772838 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b5079-ca62-48a3-9888-3505f9c9ff5b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:07.800889 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1260/2513] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:40:07 2022] POST /identity/v3/projects => generated 405 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:40:07.805131 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479eca5-3b42-45d7-9936-d73aa01d289c None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:07.817725 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479eca5-3b42-45d7-9936-d73aa01d289c None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:07.879669 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cada22a-3160-4c36-b567-273294632b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:07.880449 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cada22a-3160-4c36-b567-273294632b92 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:07.880730 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cada22a-3160-4c36-b567-273294632b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:07.880998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cada22a-3160-4c36-b567-273294632b92 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:07.881705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cada22a-3160-4c36-b567-273294632b92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:07.894434 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cada22a-3160-4c36-b567-273294632b92 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:07.905298 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:07.923223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1254/2514] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:40:07 2022] POST /identity/v3/users => generated 309 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:40:07.927715 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b5079-ca62-48a3-9888-3505f9c9ff5b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:07.940024 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b5079-ca62-48a3-9888-3505f9c9ff5b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:07.944822 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb4281f-ff99-49c9-947d-543bc937ba71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:07.946090 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4281f-ff99-49c9-947d-543bc937ba71 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:07.946405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4281f-ff99-49c9-947d-543bc937ba71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:07.946686 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4281f-ff99-49c9-947d-543bc937ba71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:07.947342 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb4281f-ff99-49c9-947d-543bc937ba71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:07.949491 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb4281f-ff99-49c9-947d-543bc937ba71 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:07.960947 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1261/2515] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:40:07.964906 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cada22a-3160-4c36-b567-273294632b92 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:07.980080 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cada22a-3160-4c36-b567-273294632b92 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:07.986340 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf04536-d1d6-4ba9-b29f-ff5468890767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:07.987127 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf04536-d1d6-4ba9-b29f-ff5468890767 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:07.987640 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf04536-d1d6-4ba9-b29f-ff5468890767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:07.988015 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf04536-d1d6-4ba9-b29f-ff5468890767 admin admin] PATH_INFO: `/v3/projects/479556cd676d4e9592f6f348f8a701e5/users/7bde456833844e169ac267f0a327b8a8/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:08.006454 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf04536-d1d6-4ba9-b29f-ff5468890767 admin admin] RBAC: Authorizing `identity:create_grant(project_id=479556cd676d4e9592f6f348f8a701e5, user_id=7bde456833844e169ac267f0a327b8a8, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:08.007701 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf04536-d1d6-4ba9-b29f-ff5468890767 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:08.022458 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1255/2516] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:40:07 2022] PUT /identity/v3/projects/479556cd676d4e9592f6f348f8a701e5/users/7bde456833844e169ac267f0a327b8a8/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:08.026558 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb4281f-ff99-49c9-947d-543bc937ba71 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:08.037690 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb4281f-ff99-49c9-947d-543bc937ba71 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:08.084496 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:08.098723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ddab3-3242-4377-adab-2385b377b2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:08.099340 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ddab3-3242-4377-adab-2385b377b2a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:08.099620 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ddab3-3242-4377-adab-2385b377b2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:08.099889 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ddab3-3242-4377-adab-2385b377b2a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:08.100529 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5ddab3-3242-4377-adab-2385b377b2a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:08.102224 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5ddab3-3242-4377-adab-2385b377b2a7 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:08.113286 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1262/2517] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:40:08 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:40:08.117496 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf04536-d1d6-4ba9-b29f-ff5468890767 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:08.128454 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf04536-d1d6-4ba9-b29f-ff5468890767 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:08.133330 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150834b-ebcb-41e7-9f88-18d4d851a874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:08.134001 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150834b-ebcb-41e7-9f88-18d4d851a874 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:08.134421 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150834b-ebcb-41e7-9f88-18d4d851a874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:08.135030 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150834b-ebcb-41e7-9f88-18d4d851a874 admin admin] PATH_INFO: `/v3/projects/479556cd676d4e9592f6f348f8a701e5/users/7bde456833844e169ac267f0a327b8a8/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:08.137606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8150834b-ebcb-41e7-9f88-18d4d851a874 admin admin] RBAC: Authorizing `identity:create_grant(project_id=479556cd676d4e9592f6f348f8a701e5, user_id=7bde456833844e169ac267f0a327b8a8, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:08.140329 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8150834b-ebcb-41e7-9f88-18d4d851a874 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:08.152901 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1256/2518] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:40:08 2022] PUT /identity/v3/projects/479556cd676d4e9592f6f348f8a701e5/users/7bde456833844e169ac267f0a327b8a8/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:40:08.163117 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df070369-b13b-4327-aea1-226dc736196c None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:08.163409 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df070369-b13b-4327-aea1-226dc736196c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:08.163977 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df070369-b13b-4327-aea1-226dc736196c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:08.192294 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-df070369-b13b-4327-aea1-226dc736196c None None] MFA Rules not processed for user `7bde456833844e169ac267f0a327b8a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:08.193512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-df070369-b13b-4327-aea1-226dc736196c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:08.261339 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1263/2519] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:40:08 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:08.272463 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150834b-ebcb-41e7-9f88-18d4d851a874 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:08.274257 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-8150834b-ebcb-41e7-9f88-18d4d851a874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:08.332233 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150834b-ebcb-41e7-9f88-18d4d851a874 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:08.336031 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c1967-715f-4804-9506-89ce83f34105 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:08.336631 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c1967-715f-4804-9506-89ce83f34105 service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:08.336910 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c1967-715f-4804-9506-89ce83f34105 service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:08.337181 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c1967-715f-4804-9506-89ce83f34105 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:08.346323 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c1967-715f-4804-9506-89ce83f34105 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:08.348075 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c1967-715f-4804-9506-89ce83f34105 service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:08.358798 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1257/2520] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:40:08 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:40:08.520147 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df070369-b13b-4327-aea1-226dc736196c None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:08.586681 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df070369-b13b-4327-aea1-226dc736196c None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:08.591854 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb52dfd-009d-43b9-9e1a-b90a3af0bef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:08.592493 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb52dfd-009d-43b9-9e1a-b90a3af0bef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:08.592774 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb52dfd-009d-43b9-9e1a-b90a3af0bef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:08.593045 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb52dfd-009d-43b9-9e1a-b90a3af0bef4 admin admin] PATH_INFO: `/v3/users/2fd04fb6525f4a52b8584d934a0be9ab` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:08.594176 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb52dfd-009d-43b9-9e1a-b90a3af0bef4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fd04fb6525f4a52b8584d934a0be9ab)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:08.596108 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb52dfd-009d-43b9-9e1a-b90a3af0bef4 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:08.642809 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2fb52dfd-009d-43b9-9e1a-b90a3af0bef4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fd04fb6525f4a52b8584d934a0be9ab'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:08.652980 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2fb52dfd-009d-43b9-9e1a-b90a3af0bef4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fd04fb6525f4a52b8584d934a0be9ab'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:08.654271 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2fb52dfd-009d-43b9-9e1a-b90a3af0bef4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fd04fb6525f4a52b8584d934a0be9ab'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:08.669607 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2fb52dfd-009d-43b9-9e1a-b90a3af0bef4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fd04fb6525f4a52b8584d934a0be9ab'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:08.670563 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-2fb52dfd-009d-43b9-9e1a-b90a3af0bef4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fd04fb6525f4a52b8584d934a0be9ab'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:08.671879 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:08.687303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1264/2521] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:40:08 2022] DELETE /identity/v3/users/2fd04fb6525f4a52b8584d934a0be9ab => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:08.692141 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a0bf0-50f8-40e9-a7e0-7372ec05020e None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:08.692607 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a0bf0-50f8-40e9-a7e0-7372ec05020e None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:08.692712 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a0bf0-50f8-40e9-a7e0-7372ec05020e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:08.711294 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-794a0bf0-50f8-40e9-a7e0-7372ec05020e None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:08.712498 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-794a0bf0-50f8-40e9-a7e0-7372ec05020e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:08.771827 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1258/2522] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:40:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:08.781896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb52dfd-009d-43b9-9e1a-b90a3af0bef4 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:08.783436 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2fb52dfd-009d-43b9-9e1a-b90a3af0bef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:08.844462 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb52dfd-009d-43b9-9e1a-b90a3af0bef4 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:08.848652 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8171002f-e044-4894-959f-867d8d1e58e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:08.849267 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171002f-e044-4894-959f-867d8d1e58e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:08.849579 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171002f-e044-4894-959f-867d8d1e58e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:08.849858 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171002f-e044-4894-959f-867d8d1e58e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:08.859777 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8171002f-e044-4894-959f-867d8d1e58e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:08.861207 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8171002f-e044-4894-959f-867d8d1e58e5 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:08.870654 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1265/2523] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:40:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:40:09.203444 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a0bf0-50f8-40e9-a7e0-7372ec05020e None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:09.205066 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-794a0bf0-50f8-40e9-a7e0-7372ec05020e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:09.221714 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a0bf0-50f8-40e9-a7e0-7372ec05020e None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:09.226755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6630b24-1ed5-4e1a-808e-2d5dbb0ab521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:09.228076 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6630b24-1ed5-4e1a-808e-2d5dbb0ab521 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:09.228076 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6630b24-1ed5-4e1a-808e-2d5dbb0ab521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:09.228076 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6630b24-1ed5-4e1a-808e-2d5dbb0ab521 admin admin] PATH_INFO: `/v3/projects/ec6cd5c5d62643c2898f6d7188f66252` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:09.229086 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6630b24-1ed5-4e1a-808e-2d5dbb0ab521 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec6cd5c5d62643c2898f6d7188f66252)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:09.231064 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6630b24-1ed5-4e1a-808e-2d5dbb0ab521 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:09.284538 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c6630b24-1ed5-4e1a-808e-2d5dbb0ab521 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec6cd5c5d62643c2898f6d7188f66252', 'request_id': 'req-c6630b24-1ed5-4e1a-808e-2d5dbb0ab521'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:09.287406 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:09.292302 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c6630b24-1ed5-4e1a-808e-2d5dbb0ab521 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec6cd5c5d62643c2898f6d7188f66252', 'request_id': 'req-c6630b24-1ed5-4e1a-808e-2d5dbb0ab521'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:09.299886 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c6630b24-1ed5-4e1a-808e-2d5dbb0ab521 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec6cd5c5d62643c2898f6d7188f66252', 'request_id': 'req-c6630b24-1ed5-4e1a-808e-2d5dbb0ab521'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:09.309993 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c6630b24-1ed5-4e1a-808e-2d5dbb0ab521 admin admin] The token cache is being invalidate because project ec6cd5c5d62643c2898f6d7188f66252 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:40:09.310317 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c6630b24-1ed5-4e1a-808e-2d5dbb0ab521 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:09.310943 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c6630b24-1ed5-4e1a-808e-2d5dbb0ab521 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:09.312405 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1259/2524] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:40:09 2022] DELETE /identity/v3/projects/ec6cd5c5d62643c2898f6d7188f66252 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:09.316597 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8171002f-e044-4894-959f-867d8d1e58e5 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:09.326818 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8171002f-e044-4894-959f-867d8d1e58e5 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:09.399803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f98a1f5-3bb4-4b0f-969f-ea8640cb23e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:09.400441 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98a1f5-3bb4-4b0f-969f-ea8640cb23e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:09.400724 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98a1f5-3bb4-4b0f-969f-ea8640cb23e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:09.400997 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98a1f5-3bb4-4b0f-969f-ea8640cb23e0 admin admin] PATH_INFO: `/v3/users/7bde456833844e169ac267f0a327b8a8` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:09.402124 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f98a1f5-3bb4-4b0f-969f-ea8640cb23e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bde456833844e169ac267f0a327b8a8)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:09.404789 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f98a1f5-3bb4-4b0f-969f-ea8640cb23e0 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:09.459004 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1f98a1f5-3bb4-4b0f-969f-ea8640cb23e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bde456833844e169ac267f0a327b8a8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:09.469659 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1f98a1f5-3bb4-4b0f-969f-ea8640cb23e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bde456833844e169ac267f0a327b8a8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:09.470662 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1f98a1f5-3bb4-4b0f-969f-ea8640cb23e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bde456833844e169ac267f0a327b8a8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:09.485388 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1f98a1f5-3bb4-4b0f-969f-ea8640cb23e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bde456833844e169ac267f0a327b8a8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:09.486186 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1f98a1f5-3bb4-4b0f-969f-ea8640cb23e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bde456833844e169ac267f0a327b8a8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:09.505124 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1266/2525] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:40:09 2022] DELETE /identity/v3/users/7bde456833844e169ac267f0a327b8a8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:09.781159 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6630b24-1ed5-4e1a-808e-2d5dbb0ab521 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:09.782864 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-c6630b24-1ed5-4e1a-808e-2d5dbb0ab521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:09.798042 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6630b24-1ed5-4e1a-808e-2d5dbb0ab521 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:09.857905 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8a9251-04d5-4daf-a1a1-9c86ff29e711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:09.858531 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a9251-04d5-4daf-a1a1-9c86ff29e711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:09.858809 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a9251-04d5-4daf-a1a1-9c86ff29e711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:09.859164 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a9251-04d5-4daf-a1a1-9c86ff29e711 admin admin] PATH_INFO: `/v3/projects/479556cd676d4e9592f6f348f8a701e5` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:09.860412 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8a9251-04d5-4daf-a1a1-9c86ff29e711 admin admin] RBAC: Authorizing `identity:delete_project(project_id=479556cd676d4e9592f6f348f8a701e5)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:09.862214 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8a9251-04d5-4daf-a1a1-9c86ff29e711 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:09.865631 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183643b7-5d85-486b-859e-070e189889d3 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:09.865926 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183643b7-5d85-486b-859e-070e189889d3 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:09.866210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183643b7-5d85-486b-859e-070e189889d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:09.889905 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-183643b7-5d85-486b-859e-070e189889d3 None None] MFA Rules not processed for user `a5b3e53fd1314e17ad28c42a44e8eff3`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:09.891255 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-183643b7-5d85-486b-859e-070e189889d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:09.897112 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:09.916410 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-6a8a9251-04d5-4daf-a1a1-9c86ff29e711 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '479556cd676d4e9592f6f348f8a701e5', 'request_id': 'req-6a8a9251-04d5-4daf-a1a1-9c86ff29e711'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:09.924367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-6a8a9251-04d5-4daf-a1a1-9c86ff29e711 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '479556cd676d4e9592f6f348f8a701e5', 'request_id': 'req-6a8a9251-04d5-4daf-a1a1-9c86ff29e711'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:09.931553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-6a8a9251-04d5-4daf-a1a1-9c86ff29e711 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '479556cd676d4e9592f6f348f8a701e5', 'request_id': 'req-6a8a9251-04d5-4daf-a1a1-9c86ff29e711'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:09.942053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-6a8a9251-04d5-4daf-a1a1-9c86ff29e711 admin admin] The token cache is being invalidate because project 479556cd676d4e9592f6f348f8a701e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:40:09.942378 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-6a8a9251-04d5-4daf-a1a1-9c86ff29e711 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:09.943022 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-6a8a9251-04d5-4daf-a1a1-9c86ff29e711 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:09.944482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1260/2526] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:40:09 2022] DELETE /identity/v3/projects/479556cd676d4e9592f6f348f8a701e5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:09.950881 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32eb15-2729-47ba-a525-efa243010534 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:09.951253 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32eb15-2729-47ba-a525-efa243010534 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:09.951532 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32eb15-2729-47ba-a525-efa243010534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:09.952873 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1267/2527] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Sep 1 16:40:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:40:09.971955 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-8c32eb15-2729-47ba-a525-efa243010534 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:09.973564 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-8c32eb15-2729-47ba-a525-efa243010534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:10.001060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183643b7-5d85-486b-859e-070e189889d3 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:10.002577 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-183643b7-5d85-486b-859e-070e189889d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:10.035434 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1261/2528] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:40:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:40:10.042979 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55199e96-e2e9-4513-a74e-1a4c49fc6e45 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:10.043295 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55199e96-e2e9-4513-a74e-1a4c49fc6e45 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:10.043568 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55199e96-e2e9-4513-a74e-1a4c49fc6e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:10.063094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-55199e96-e2e9-4513-a74e-1a4c49fc6e45 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:10.064322 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-55199e96-e2e9-4513-a74e-1a4c49fc6e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:10.072366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1262/2529] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:40:10.076936 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55199e96-e2e9-4513-a74e-1a4c49fc6e45 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:10.077445 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183643b7-5d85-486b-859e-070e189889d3 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:10.082452 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:10.086369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55199e96-e2e9-4513-a74e-1a4c49fc6e45 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:10.093548 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77840d62-d892-4a71-8ced-0d4438e648ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:10.094323 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77840d62-d892-4a71-8ced-0d4438e648ae admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:10.094780 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77840d62-d892-4a71-8ced-0d4438e648ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:10.095262 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77840d62-d892-4a71-8ced-0d4438e648ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:10.095959 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77840d62-d892-4a71-8ced-0d4438e648ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:40:10.096417 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77840d62-d892-4a71-8ced-0d4438e648ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:10.098344 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77840d62-d892-4a71-8ced-0d4438e648ae admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:10.113564 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1263/2530] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:40:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:40:10.114511 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77840d62-d892-4a71-8ced-0d4438e648ae admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:10.124102 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77840d62-d892-4a71-8ced-0d4438e648ae admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:10.128645 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5df118-4959-4291-b699-3414ab0ab20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:10.129229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5df118-4959-4291-b699-3414ab0ab20d admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:10.129512 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5df118-4959-4291-b699-3414ab0ab20d admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:10.129808 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5df118-4959-4291-b699-3414ab0ab20d admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:10.130474 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5df118-4959-4291-b699-3414ab0ab20d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:10.132375 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5df118-4959-4291-b699-3414ab0ab20d admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:10.142169 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cebd47-4b3c-4081-a52f-48edbb60860f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8137f6b2ced040bd9752239c7e987f03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:10.143084 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cebd47-4b3c-4081-a52f-48edbb60860f service swift] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:10.143366 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cebd47-4b3c-4081-a52f-48edbb60860f service swift] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:10.143637 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cebd47-4b3c-4081-a52f-48edbb60860f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:10.163986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1264/2531] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:40:10 2022] POST /identity/v3/projects => generated 401 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:40:10.168249 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5df118-4959-4291-b699-3414ab0ab20d admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:10.220548 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cebd47-4b3c-4081-a52f-48edbb60860f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:10.222172 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cebd47-4b3c-4081-a52f-48edbb60860f service swift] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:10.230706 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:10.240213 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1268/2532] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 1 16:40:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 16:40:10.257973 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5df118-4959-4291-b699-3414ab0ab20d admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:10.263163 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f93af1f-d647-4153-a837-2a5258c86b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:10.263859 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93af1f-d647-4153-a837-2a5258c86b17 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:10.264222 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93af1f-d647-4153-a837-2a5258c86b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:10.264574 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93af1f-d647-4153-a837-2a5258c86b17 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:10.265292 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f93af1f-d647-4153-a837-2a5258c86b17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:10.267395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f93af1f-d647-4153-a837-2a5258c86b17 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:10.302878 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cebd47-4b3c-4081-a52f-48edbb60860f service swift] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:10.306929 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1265/2533] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:40:10 2022] POST /identity/v3/users => generated 301 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:40:10.317393 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cebd47-4b3c-4081-a52f-48edbb60860f service swift] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:10.340062 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a5de1a-d21a-4169-be79-f9b3e4cf341d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:10.340935 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5de1a-d21a-4169-be79-f9b3e4cf341d admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:10.341524 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5de1a-d21a-4169-be79-f9b3e4cf341d admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:10.342010 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5de1a-d21a-4169-be79-f9b3e4cf341d admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:10.343419 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a5de1a-d21a-4169-be79-f9b3e4cf341d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:10.348235 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a5de1a-d21a-4169-be79-f9b3e4cf341d admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:10.361212 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1269/2534] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:40:10.369083 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f93af1f-d647-4153-a837-2a5258c86b17 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:10.383093 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f93af1f-d647-4153-a837-2a5258c86b17 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:10.393823 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d134e5-2241-4d3e-b85d-0c48e737ee9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:10.395590 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d134e5-2241-4d3e-b85d-0c48e737ee9d admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:10.395870 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d134e5-2241-4d3e-b85d-0c48e737ee9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:10.396143 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d134e5-2241-4d3e-b85d-0c48e737ee9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:10.399105 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d134e5-2241-4d3e-b85d-0c48e737ee9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:10.400914 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d134e5-2241-4d3e-b85d-0c48e737ee9d admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:10.414541 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1266/2535] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:40:10.418345 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a5de1a-d21a-4169-be79-f9b3e4cf341d admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:10.431923 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a5de1a-d21a-4169-be79-f9b3e4cf341d admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:10.457358 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910b30a-6120-495b-a6e9-601688e14d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:10.458172 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910b30a-6120-495b-a6e9-601688e14d86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:10.458483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910b30a-6120-495b-a6e9-601688e14d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:10.458763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910b30a-6120-495b-a6e9-601688e14d86 admin admin] PATH_INFO: `/v3/projects/7a48b83ad43347a0898a5bd01ca224ee/users/ecf0a3441e124984bf8affb223778567/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:10.481944 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a910b30a-6120-495b-a6e9-601688e14d86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a48b83ad43347a0898a5bd01ca224ee, user_id=ecf0a3441e124984bf8affb223778567, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:10.484806 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a910b30a-6120-495b-a6e9-601688e14d86 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:10.501002 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1270/2536] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:40:10 2022] PUT /identity/v3/projects/7a48b83ad43347a0898a5bd01ca224ee/users/ecf0a3441e124984bf8affb223778567/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:10.512990 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df21b6-0acf-46e3-a1d1-cab0ddd89f83 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:10.513284 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df21b6-0acf-46e3-a1d1-cab0ddd89f83 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:10.513566 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df21b6-0acf-46e3-a1d1-cab0ddd89f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:10.545730 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-07df21b6-0acf-46e3-a1d1-cab0ddd89f83 None None] MFA Rules not processed for user `ecf0a3441e124984bf8affb223778567`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:10.547159 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-07df21b6-0acf-46e3-a1d1-cab0ddd89f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:10.647098 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1267/2537] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:40:10 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:10.651369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910b30a-6120-495b-a6e9-601688e14d86 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:10.706530 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df21b6-0acf-46e3-a1d1-cab0ddd89f83 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:10.708434 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-07df21b6-0acf-46e3-a1d1-cab0ddd89f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:10.784962 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:10.790642 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910b30a-6120-495b-a6e9-601688e14d86 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:10.800644 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee104164-56fd-47c4-a76c-d6c5c3f1cdba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:10.801277 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee104164-56fd-47c4-a76c-d6c5c3f1cdba admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:10.801559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee104164-56fd-47c4-a76c-d6c5c3f1cdba admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:10.802128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee104164-56fd-47c4-a76c-d6c5c3f1cdba admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:10.804063 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee104164-56fd-47c4-a76c-d6c5c3f1cdba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:10.806444 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee104164-56fd-47c4-a76c-d6c5c3f1cdba admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:10.842805 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df21b6-0acf-46e3-a1d1-cab0ddd89f83 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:10.856821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4e3bf-fff0-4e5e-b5a1-9327424f3e78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:10.856821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4e3bf-fff0-4e5e-b5a1-9327424f3e78 service nova] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:10.856821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4e3bf-fff0-4e5e-b5a1-9327424f3e78 service nova] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:10.856821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4e3bf-fff0-4e5e-b5a1-9327424f3e78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:10.857302 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1271/2538] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:40:10 2022] POST /identity/v3/projects => generated 401 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:40:10.858189 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee104164-56fd-47c4-a76c-d6c5c3f1cdba admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:10.859649 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-f3b4e3bf-fff0-4e5e-b5a1-9327424f3e78 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:10.882820 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4e3bf-fff0-4e5e-b5a1-9327424f3e78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:10.885846 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4e3bf-fff0-4e5e-b5a1-9327424f3e78 service nova] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:10.918598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1268/2539] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:40:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:40:10.990498 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee104164-56fd-47c4-a76c-d6c5c3f1cdba admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:10.996800 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0bdc06-7821-4dda-a472-fd7b6e7d9dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:10.997431 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0bdc06-7821-4dda-a472-fd7b6e7d9dc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:10.997714 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0bdc06-7821-4dda-a472-fd7b6e7d9dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:10.997984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0bdc06-7821-4dda-a472-fd7b6e7d9dc5 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:10.998652 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0bdc06-7821-4dda-a472-fd7b6e7d9dc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:11.000981 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0bdc06-7821-4dda-a472-fd7b6e7d9dc5 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:11.032553 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1272/2540] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:40:10 2022] POST /identity/v3/users => generated 307 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:40:11.033694 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4e3bf-fff0-4e5e-b5a1-9327424f3e78 service nova] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:11.049417 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4e3bf-fff0-4e5e-b5a1-9327424f3e78 service nova] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:11.053895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e20136-0634-4b5a-9b3a-c29446a60519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:11.054524 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e20136-0634-4b5a-9b3a-c29446a60519 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:11.054805 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e20136-0634-4b5a-9b3a-c29446a60519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:11.055083 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e20136-0634-4b5a-9b3a-c29446a60519 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:11.055726 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e20136-0634-4b5a-9b3a-c29446a60519 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:11.057321 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e20136-0634-4b5a-9b3a-c29446a60519 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:11.072694 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1269/2541] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:40:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:40:11.077424 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0bdc06-7821-4dda-a472-fd7b6e7d9dc5 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:11.087945 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0bdc06-7821-4dda-a472-fd7b6e7d9dc5 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:11.093692 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0db3f3-71eb-4653-9e71-2b274a0c8f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:11.094454 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0db3f3-71eb-4653-9e71-2b274a0c8f90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:11.094744 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0db3f3-71eb-4653-9e71-2b274a0c8f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:11.095687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0db3f3-71eb-4653-9e71-2b274a0c8f90 admin admin] PATH_INFO: `/v3/projects/9b29905dc69e4cd6ba264eaf73188891/users/a615d8bb5cb44fa9932a337b78741ae0/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:11.113064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0db3f3-71eb-4653-9e71-2b274a0c8f90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b29905dc69e4cd6ba264eaf73188891, user_id=a615d8bb5cb44fa9932a337b78741ae0, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:11.115786 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0db3f3-71eb-4653-9e71-2b274a0c8f90 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:11.134857 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1273/2542] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:40:11 2022] PUT /identity/v3/projects/9b29905dc69e4cd6ba264eaf73188891/users/a615d8bb5cb44fa9932a337b78741ae0/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:40:11.140221 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e20136-0634-4b5a-9b3a-c29446a60519 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:11.163182 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e20136-0634-4b5a-9b3a-c29446a60519 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:11.241530 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae3d95-49a3-4c8c-92d9-d5afd30e80e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:11.242298 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae3d95-49a3-4c8c-92d9-d5afd30e80e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:11.242652 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae3d95-49a3-4c8c-92d9-d5afd30e80e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:11.243108 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae3d95-49a3-4c8c-92d9-d5afd30e80e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:11.243761 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ae3d95-49a3-4c8c-92d9-d5afd30e80e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:11.245409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ae3d95-49a3-4c8c-92d9-d5afd30e80e1 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:11.259030 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1270/2543] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:40:11 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:40:11.264165 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0db3f3-71eb-4653-9e71-2b274a0c8f90 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:11.278382 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0db3f3-71eb-4653-9e71-2b274a0c8f90 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:11.283210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636f19a-6733-4a98-b985-8cece51a6a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:11.283810 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636f19a-6733-4a98-b985-8cece51a6a66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:11.284089 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636f19a-6733-4a98-b985-8cece51a6a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:11.284363 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636f19a-6733-4a98-b985-8cece51a6a66 admin admin] PATH_INFO: `/v3/projects/9b29905dc69e4cd6ba264eaf73188891/users/a615d8bb5cb44fa9932a337b78741ae0/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:11.286303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636f19a-6733-4a98-b985-8cece51a6a66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b29905dc69e4cd6ba264eaf73188891, user_id=a615d8bb5cb44fa9932a337b78741ae0, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:11.291910 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636f19a-6733-4a98-b985-8cece51a6a66 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:11.304060 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1274/2544] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:40:11 2022] PUT /identity/v3/projects/9b29905dc69e4cd6ba264eaf73188891/users/a615d8bb5cb44fa9932a337b78741ae0/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:11.314604 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98733ce5-fd23-4ad1-b36d-440174ab4bc4 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:11.314890 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98733ce5-fd23-4ad1-b36d-440174ab4bc4 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:11.315166 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98733ce5-fd23-4ad1-b36d-440174ab4bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:11.363058 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-98733ce5-fd23-4ad1-b36d-440174ab4bc4 None None] MFA Rules not processed for user `a615d8bb5cb44fa9932a337b78741ae0`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:11.366271 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-98733ce5-fd23-4ad1-b36d-440174ab4bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:11.482027 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1271/2545] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:40:11 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:11.489885 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636f19a-6733-4a98-b985-8cece51a6a66 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:11.492839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c636f19a-6733-4a98-b985-8cece51a6a66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:11.502342 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:11.602920 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636f19a-6733-4a98-b985-8cece51a6a66 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:11.609659 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d227b29-877f-4f92-a9ef-9eeaddf226dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:11.610319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d227b29-877f-4f92-a9ef-9eeaddf226dc service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:11.610603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d227b29-877f-4f92-a9ef-9eeaddf226dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:11.611021 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d227b29-877f-4f92-a9ef-9eeaddf226dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:11.624336 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d227b29-877f-4f92-a9ef-9eeaddf226dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:11.625782 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d227b29-877f-4f92-a9ef-9eeaddf226dc service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:11.638905 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1275/2546] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:40:11 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:40:11.748723 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98733ce5-fd23-4ad1-b36d-440174ab4bc4 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:11.763933 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98733ce5-fd23-4ad1-b36d-440174ab4bc4 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:11.767917 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743567cd-88e0-4aa5-84da-c30dbe5dd9a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:11.768759 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743567cd-88e0-4aa5-84da-c30dbe5dd9a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:11.769353 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743567cd-88e0-4aa5-84da-c30dbe5dd9a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:11.769920 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743567cd-88e0-4aa5-84da-c30dbe5dd9a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:11.807194 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:11.840642 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743567cd-88e0-4aa5-84da-c30dbe5dd9a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:11.842048 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743567cd-88e0-4aa5-84da-c30dbe5dd9a6 service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:11.853240 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1272/2547] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:40:11 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:40:16.349804 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d227b29-877f-4f92-a9ef-9eeaddf226dc service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:16.499686 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d227b29-877f-4f92-a9ef-9eeaddf226dc service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:16.505733 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452dd782-9d7b-4f8c-bcea-91ded45b393b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:16.506687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452dd782-9d7b-4f8c-bcea-91ded45b393b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:16.507143 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452dd782-9d7b-4f8c-bcea-91ded45b393b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:16.507561 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452dd782-9d7b-4f8c-bcea-91ded45b393b admin admin] PATH_INFO: `/v3/users/ecf0a3441e124984bf8affb223778567` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:16.509442 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452dd782-9d7b-4f8c-bcea-91ded45b393b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecf0a3441e124984bf8affb223778567)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:16.513581 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452dd782-9d7b-4f8c-bcea-91ded45b393b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:16.593177 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-452dd782-9d7b-4f8c-bcea-91ded45b393b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecf0a3441e124984bf8affb223778567'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:16.603477 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-452dd782-9d7b-4f8c-bcea-91ded45b393b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecf0a3441e124984bf8affb223778567'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:16.604294 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-452dd782-9d7b-4f8c-bcea-91ded45b393b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecf0a3441e124984bf8affb223778567'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:16.618745 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-452dd782-9d7b-4f8c-bcea-91ded45b393b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecf0a3441e124984bf8affb223778567'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:16.619484 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-452dd782-9d7b-4f8c-bcea-91ded45b393b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecf0a3441e124984bf8affb223778567'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:16.632855 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1276/2548] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:40:16 2022] DELETE /identity/v3/users/ecf0a3441e124984bf8affb223778567 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:16.637600 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadc46e-1861-4e12-af8d-811c5fa74306 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:16.638164 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadc46e-1861-4e12-af8d-811c5fa74306 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:16.638164 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadc46e-1861-4e12-af8d-811c5fa74306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:16.662339 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-7eadc46e-1861-4e12-af8d-811c5fa74306 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:16.663900 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-7eadc46e-1861-4e12-af8d-811c5fa74306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:16.773092 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1273/2549] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:40:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:16.781205 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452dd782-9d7b-4f8c-bcea-91ded45b393b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:16.782754 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-452dd782-9d7b-4f8c-bcea-91ded45b393b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:16.823096 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:16.866572 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452dd782-9d7b-4f8c-bcea-91ded45b393b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:16.870256 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42adf6ca-b62e-4fcf-b2f0-521ebbf87d2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:16.870971 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42adf6ca-b62e-4fcf-b2f0-521ebbf87d2e service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:16.871743 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42adf6ca-b62e-4fcf-b2f0-521ebbf87d2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:16.872016 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42adf6ca-b62e-4fcf-b2f0-521ebbf87d2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:16.883477 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42adf6ca-b62e-4fcf-b2f0-521ebbf87d2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:16.885393 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42adf6ca-b62e-4fcf-b2f0-521ebbf87d2e service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:16.899941 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1277/2550] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:40:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:40:17.401589 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eadc46e-1861-4e12-af8d-811c5fa74306 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:17.404334 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-7eadc46e-1861-4e12-af8d-811c5fa74306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:17.424387 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eadc46e-1861-4e12-af8d-811c5fa74306 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:17.436580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae475eb-8195-413d-8ae2-917de992d5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:17.437388 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae475eb-8195-413d-8ae2-917de992d5ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:17.437825 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae475eb-8195-413d-8ae2-917de992d5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:17.439433 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae475eb-8195-413d-8ae2-917de992d5ae admin admin] PATH_INFO: `/v3/projects/7a48b83ad43347a0898a5bd01ca224ee` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:17.440843 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae475eb-8195-413d-8ae2-917de992d5ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a48b83ad43347a0898a5bd01ca224ee)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:17.443117 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae475eb-8195-413d-8ae2-917de992d5ae admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:17.550827 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-8ae475eb-8195-413d-8ae2-917de992d5ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a48b83ad43347a0898a5bd01ca224ee', 'request_id': 'req-8ae475eb-8195-413d-8ae2-917de992d5ae'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:17.562550 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-8ae475eb-8195-413d-8ae2-917de992d5ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a48b83ad43347a0898a5bd01ca224ee', 'request_id': 'req-8ae475eb-8195-413d-8ae2-917de992d5ae'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:17.572144 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-8ae475eb-8195-413d-8ae2-917de992d5ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a48b83ad43347a0898a5bd01ca224ee', 'request_id': 'req-8ae475eb-8195-413d-8ae2-917de992d5ae'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:17.586529 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-8ae475eb-8195-413d-8ae2-917de992d5ae admin admin] The token cache is being invalidate because project 7a48b83ad43347a0898a5bd01ca224ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:40:17.586840 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-8ae475eb-8195-413d-8ae2-917de992d5ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:17.587480 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-8ae475eb-8195-413d-8ae2-917de992d5ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:17.592523 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1274/2551] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:40:17 2022] DELETE /identity/v3/projects/7a48b83ad43347a0898a5bd01ca224ee => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:17.595835 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42adf6ca-b62e-4fcf-b2f0-521ebbf87d2e service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:17.607409 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42adf6ca-b62e-4fcf-b2f0-521ebbf87d2e service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:17.687494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d97add-c79a-47a9-ac2b-8a9b18716cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:17.688143 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d97add-c79a-47a9-ac2b-8a9b18716cc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:17.688498 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d97add-c79a-47a9-ac2b-8a9b18716cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:17.688779 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d97add-c79a-47a9-ac2b-8a9b18716cc6 admin admin] PATH_INFO: `/v3/users/a615d8bb5cb44fa9932a337b78741ae0` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:17.692209 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d97add-c79a-47a9-ac2b-8a9b18716cc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a615d8bb5cb44fa9932a337b78741ae0)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:17.694178 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d97add-c79a-47a9-ac2b-8a9b18716cc6 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:17.808579 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-a1d97add-c79a-47a9-ac2b-8a9b18716cc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a615d8bb5cb44fa9932a337b78741ae0'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:17.821989 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-a1d97add-c79a-47a9-ac2b-8a9b18716cc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a615d8bb5cb44fa9932a337b78741ae0'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:17.830281 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-a1d97add-c79a-47a9-ac2b-8a9b18716cc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a615d8bb5cb44fa9932a337b78741ae0'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:17.866449 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-a1d97add-c79a-47a9-ac2b-8a9b18716cc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a615d8bb5cb44fa9932a337b78741ae0'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:17.867229 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-a1d97add-c79a-47a9-ac2b-8a9b18716cc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a615d8bb5cb44fa9932a337b78741ae0'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:17.901672 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1278/2552] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:40:17 2022] DELETE /identity/v3/users/a615d8bb5cb44fa9932a337b78741ae0 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:40:18.168417 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae475eb-8195-413d-8ae2-917de992d5ae admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:18.170038 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-8ae475eb-8195-413d-8ae2-917de992d5ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:18.251936 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae475eb-8195-413d-8ae2-917de992d5ae admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:18.271651 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:18.288898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d97add-c79a-47a9-ac2b-8a9b18716cc6 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:18.290403 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a1d97add-c79a-47a9-ac2b-8a9b18716cc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:18.346715 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d329ae-5e75-480a-9186-b6bc274baf09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:18.355692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d329ae-5e75-480a-9186-b6bc274baf09 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:18.355987 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d329ae-5e75-480a-9186-b6bc274baf09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:18.356265 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d329ae-5e75-480a-9186-b6bc274baf09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:18.359054 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-53d329ae-5e75-480a-9186-b6bc274baf09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:18.391137 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:18.417532 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d97add-c79a-47a9-ac2b-8a9b18716cc6 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:18.423887 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8033e-ace8-41f7-a8f3-3e013dd29216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:18.424748 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8033e-ace8-41f7-a8f3-3e013dd29216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:18.425208 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8033e-ace8-41f7-a8f3-3e013dd29216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:18.425634 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8033e-ace8-41f7-a8f3-3e013dd29216 admin admin] PATH_INFO: `/v3/projects/9b29905dc69e4cd6ba264eaf73188891` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:18.427105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a8033e-ace8-41f7-a8f3-3e013dd29216 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b29905dc69e4cd6ba264eaf73188891)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:18.429795 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a8033e-ace8-41f7-a8f3-3e013dd29216 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:18.430102 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d329ae-5e75-480a-9186-b6bc274baf09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:18.431566 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d329ae-5e75-480a-9186-b6bc274baf09 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:18.454733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1275/2553] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:40:18 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:40:18.525249 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-38a8033e-ace8-41f7-a8f3-3e013dd29216 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b29905dc69e4cd6ba264eaf73188891', 'request_id': 'req-38a8033e-ace8-41f7-a8f3-3e013dd29216'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:18.539206 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-38a8033e-ace8-41f7-a8f3-3e013dd29216 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b29905dc69e4cd6ba264eaf73188891', 'request_id': 'req-38a8033e-ace8-41f7-a8f3-3e013dd29216'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:18.560295 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-38a8033e-ace8-41f7-a8f3-3e013dd29216 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b29905dc69e4cd6ba264eaf73188891', 'request_id': 'req-38a8033e-ace8-41f7-a8f3-3e013dd29216'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:18.579013 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-38a8033e-ace8-41f7-a8f3-3e013dd29216 admin admin] The token cache is being invalidate because project 9b29905dc69e4cd6ba264eaf73188891 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:40:18.579260 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-38a8033e-ace8-41f7-a8f3-3e013dd29216 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:18.580794 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-38a8033e-ace8-41f7-a8f3-3e013dd29216 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:18.582905 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1279/2554] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:40:18 2022] DELETE /identity/v3/projects/9b29905dc69e4cd6ba264eaf73188891 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:40:18.588293 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8cc2f-375e-4f01-8387-c14268d6fe1f None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:18.588818 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8cc2f-375e-4f01-8387-c14268d6fe1f None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:18.589064 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8cc2f-375e-4f01-8387-c14268d6fe1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:18.618284 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-29b8cc2f-375e-4f01-8387-c14268d6fe1f None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:18.620013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-29b8cc2f-375e-4f01-8387-c14268d6fe1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:18.698692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1276/2555] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:40:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:18.701555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869770b-264f-4fa2-9373-5385c2a447c1 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:18.702080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869770b-264f-4fa2-9373-5385c2a447c1 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:18.702179 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869770b-264f-4fa2-9373-5385c2a447c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:18.731064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-3869770b-264f-4fa2-9373-5385c2a447c1 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:18.732770 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3869770b-264f-4fa2-9373-5385c2a447c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:18.747816 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1280/2556] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:40:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:18.751855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b8cc2f-375e-4f01-8387-c14268d6fe1f None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:18.762925 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b8cc2f-375e-4f01-8387-c14268d6fe1f None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:18.784650 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f9fcb-9ffb-404e-9677-dcee3e55aa8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:18.785640 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f9fcb-9ffb-404e-9677-dcee3e55aa8b admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:18.786055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f9fcb-9ffb-404e-9677-dcee3e55aa8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:18.786532 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f9fcb-9ffb-404e-9677-dcee3e55aa8b admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:18.787580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f9fcb-9ffb-404e-9677-dcee3e55aa8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:40:18.788072 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f9fcb-9ffb-404e-9677-dcee3e55aa8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:18.789946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f9fcb-9ffb-404e-9677-dcee3e55aa8b admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:18.810257 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1277/2557] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:40:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:40:18.811359 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869770b-264f-4fa2-9373-5385c2a447c1 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:18.823541 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869770b-264f-4fa2-9373-5385c2a447c1 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:18.842941 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f9045-0bf2-4f1e-877b-67f6d349429b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:18.843989 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f9045-0bf2-4f1e-877b-67f6d349429b admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:18.843989 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f9045-0bf2-4f1e-877b-67f6d349429b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:18.844461 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f9045-0bf2-4f1e-877b-67f6d349429b admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:18.845285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8f9045-0bf2-4f1e-877b-67f6d349429b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:18.847400 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8f9045-0bf2-4f1e-877b-67f6d349429b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:18.884448 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1281/2558] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:40:18 2022] POST /identity/v3/projects => generated 375 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:40:18.888792 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f9fcb-9ffb-404e-9677-dcee3e55aa8b admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:18.903745 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f9fcb-9ffb-404e-9677-dcee3e55aa8b admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:18.971516 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:18.993307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4a54b-e9b8-4491-ba8e-222e469cc291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:18.994102 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4a54b-e9b8-4491-ba8e-222e469cc291 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:18.994483 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4a54b-e9b8-4491-ba8e-222e469cc291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:18.994980 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4a54b-e9b8-4491-ba8e-222e469cc291 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:18.996609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff4a54b-e9b8-4491-ba8e-222e469cc291 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:19.000488 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff4a54b-e9b8-4491-ba8e-222e469cc291 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:19.029034 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1278/2559] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:40:18 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:40:19.031325 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f9045-0bf2-4f1e-877b-67f6d349429b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:19.047045 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f9045-0bf2-4f1e-877b-67f6d349429b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:19.052560 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4609e1-5446-4b51-baa5-adc4deb2676c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:19.053356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4609e1-5446-4b51-baa5-adc4deb2676c admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:19.053713 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4609e1-5446-4b51-baa5-adc4deb2676c admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:19.054268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4609e1-5446-4b51-baa5-adc4deb2676c admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:19.055166 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4609e1-5446-4b51-baa5-adc4deb2676c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:19.056947 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4609e1-5446-4b51-baa5-adc4deb2676c admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:19.077684 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1282/2560] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:40:19.078738 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4a54b-e9b8-4491-ba8e-222e469cc291 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:19.090576 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4a54b-e9b8-4491-ba8e-222e469cc291 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:19.103777 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95854e80-02f9-4214-9476-1365ceea6faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:19.104361 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95854e80-02f9-4214-9476-1365ceea6faa admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:19.104658 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95854e80-02f9-4214-9476-1365ceea6faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:19.104980 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95854e80-02f9-4214-9476-1365ceea6faa admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:19.106174 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95854e80-02f9-4214-9476-1365ceea6faa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:19.110142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95854e80-02f9-4214-9476-1365ceea6faa admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:19.139973 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1279/2561] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:40:19.141928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4609e1-5446-4b51-baa5-adc4deb2676c admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:19.166555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4609e1-5446-4b51-baa5-adc4deb2676c admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:19.179542 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b5eb8-4ea7-495d-87db-47f2fc907ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:19.181555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b5eb8-4ea7-495d-87db-47f2fc907ba2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:19.181862 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b5eb8-4ea7-495d-87db-47f2fc907ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:19.182154 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b5eb8-4ea7-495d-87db-47f2fc907ba2 admin admin] PATH_INFO: `/v3/projects/92f8156b0de74c0da84febb870b38e9b/users/1b570f2f0cf04c7e849ff6c55b33ca49/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:19.218141 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b5eb8-4ea7-495d-87db-47f2fc907ba2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92f8156b0de74c0da84febb870b38e9b, user_id=1b570f2f0cf04c7e849ff6c55b33ca49, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:19.223631 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b5eb8-4ea7-495d-87db-47f2fc907ba2 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:19.270231 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1283/2562] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:40:19 2022] PUT /identity/v3/projects/92f8156b0de74c0da84febb870b38e9b/users/1b570f2f0cf04c7e849ff6c55b33ca49/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:19.271956 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8fcf35-0302-425c-b4ff-afd75350c46b None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:19.272407 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8fcf35-0302-425c-b4ff-afd75350c46b None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:19.272687 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8fcf35-0302-425c-b4ff-afd75350c46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:19.322408 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-2a8fcf35-0302-425c-b4ff-afd75350c46b None None] MFA Rules not processed for user `1b570f2f0cf04c7e849ff6c55b33ca49`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:19.323657 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-2a8fcf35-0302-425c-b4ff-afd75350c46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:19.497820 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1280/2563] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:40:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:19.498181 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b5eb8-4ea7-495d-87db-47f2fc907ba2 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:19.633250 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b5eb8-4ea7-495d-87db-47f2fc907ba2 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:19.639766 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52a8e8-7af8-4649-9eff-59f0a859d78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:19.640627 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52a8e8-7af8-4649-9eff-59f0a859d78d admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:19.641689 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52a8e8-7af8-4649-9eff-59f0a859d78d admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:19.642190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52a8e8-7af8-4649-9eff-59f0a859d78d admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:19.657755 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a52a8e8-7af8-4649-9eff-59f0a859d78d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:19.661209 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a52a8e8-7af8-4649-9eff-59f0a859d78d admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:19.709797 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:19.723560 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1284/2564] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:40:19 2022] POST /identity/v3/projects => generated 375 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:40:19.727199 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8fcf35-0302-425c-b4ff-afd75350c46b None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:19.737944 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8fcf35-0302-425c-b4ff-afd75350c46b None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:19.802174 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b65257-7d1c-4d92-93ff-a21d6ead2832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:19.803029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b65257-7d1c-4d92-93ff-a21d6ead2832 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:19.803647 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b65257-7d1c-4d92-93ff-a21d6ead2832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:19.804268 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b65257-7d1c-4d92-93ff-a21d6ead2832 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:19.805182 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b65257-7d1c-4d92-93ff-a21d6ead2832 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:19.807984 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b65257-7d1c-4d92-93ff-a21d6ead2832 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:19.838370 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1281/2565] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:40:19 2022] POST /identity/v3/users => generated 294 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:40:19.842645 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52a8e8-7af8-4649-9eff-59f0a859d78d admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:19.856523 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52a8e8-7af8-4649-9eff-59f0a859d78d admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:19.870352 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a452313-b68b-49f3-9058-b9776d9f8957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:19.871024 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a452313-b68b-49f3-9058-b9776d9f8957 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:19.871311 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a452313-b68b-49f3-9058-b9776d9f8957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:19.871594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a452313-b68b-49f3-9058-b9776d9f8957 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:19.872245 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a452313-b68b-49f3-9058-b9776d9f8957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:19.873912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a452313-b68b-49f3-9058-b9776d9f8957 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:19.891011 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1285/2566] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:40:19.893377 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b65257-7d1c-4d92-93ff-a21d6ead2832 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:19.910437 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b65257-7d1c-4d92-93ff-a21d6ead2832 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:19.923430 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e5c41-74bd-4090-b9b6-e9ec2c20f6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:19.924669 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e5c41-74bd-4090-b9b6-e9ec2c20f6ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:19.924960 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e5c41-74bd-4090-b9b6-e9ec2c20f6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:19.925536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e5c41-74bd-4090-b9b6-e9ec2c20f6ad admin admin] PATH_INFO: `/v3/projects/dcc97eabea5740669a89ca9f4c9f98a2/users/5765cf4825b44ec7a98105476d2b6227/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:19.946500 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e5c41-74bd-4090-b9b6-e9ec2c20f6ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcc97eabea5740669a89ca9f4c9f98a2, user_id=5765cf4825b44ec7a98105476d2b6227, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:19.951189 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e5c41-74bd-4090-b9b6-e9ec2c20f6ad admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:19.969483 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1282/2567] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:40:19 2022] PUT /identity/v3/projects/dcc97eabea5740669a89ca9f4c9f98a2/users/5765cf4825b44ec7a98105476d2b6227/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:19.975691 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a452313-b68b-49f3-9058-b9776d9f8957 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:19.993307 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a452313-b68b-49f3-9058-b9776d9f8957 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:20.088203 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ccaec6-645f-4215-bc53-522483f927ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:20.088841 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ccaec6-645f-4215-bc53-522483f927ee admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:20.089128 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ccaec6-645f-4215-bc53-522483f927ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:20.089401 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ccaec6-645f-4215-bc53-522483f927ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:20.090882 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ccaec6-645f-4215-bc53-522483f927ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:20.092541 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ccaec6-645f-4215-bc53-522483f927ee admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:20.105387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1286/2568] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:40:20.115426 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e5c41-74bd-4090-b9b6-e9ec2c20f6ad admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:20.133829 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e5c41-74bd-4090-b9b6-e9ec2c20f6ad admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:20.139821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b80dd7d-519d-425d-ab6c-b2617b81902f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:20.144897 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80dd7d-519d-425d-ab6c-b2617b81902f admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:20.145110 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80dd7d-519d-425d-ab6c-b2617b81902f admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:20.145347 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80dd7d-519d-425d-ab6c-b2617b81902f admin admin] PATH_INFO: `/v3/projects/dcc97eabea5740669a89ca9f4c9f98a2/users/5765cf4825b44ec7a98105476d2b6227/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:20.147946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b80dd7d-519d-425d-ab6c-b2617b81902f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcc97eabea5740669a89ca9f4c9f98a2, user_id=5765cf4825b44ec7a98105476d2b6227, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:20.151369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b80dd7d-519d-425d-ab6c-b2617b81902f admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:20.171297 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1283/2569] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:40:20 2022] PUT /identity/v3/projects/dcc97eabea5740669a89ca9f4c9f98a2/users/5765cf4825b44ec7a98105476d2b6227/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:20.180308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17689b-58f7-4ea0-ad11-15722c2e88d1 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:20.180647 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17689b-58f7-4ea0-ad11-15722c2e88d1 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:20.180882 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17689b-58f7-4ea0-ad11-15722c2e88d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:20.217878 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-bc17689b-58f7-4ea0-ad11-15722c2e88d1 None None] MFA Rules not processed for user `5765cf4825b44ec7a98105476d2b6227`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:20.219413 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-bc17689b-58f7-4ea0-ad11-15722c2e88d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:20.317902 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1287/2570] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:40:20 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:20.331505 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b80dd7d-519d-425d-ab6c-b2617b81902f admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:20.333892 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-2b80dd7d-519d-425d-ab6c-b2617b81902f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:20.360113 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:20.456185 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b80dd7d-519d-425d-ab6c-b2617b81902f admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:20.462276 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f0f0d-1e78-48b3-bce2-1aabfad6703d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:20.463645 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f0f0d-1e78-48b3-bce2-1aabfad6703d service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:20.464045 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f0f0d-1e78-48b3-bce2-1aabfad6703d service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:20.464455 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f0f0d-1e78-48b3-bce2-1aabfad6703d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:20.580359 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303f0f0d-1e78-48b3-bce2-1aabfad6703d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:20.582048 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303f0f0d-1e78-48b3-bce2-1aabfad6703d service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:20.593415 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1284/2571] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:40:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:40:22.896043 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc17689b-58f7-4ea0-ad11-15722c2e88d1 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:22.897510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-bc17689b-58f7-4ea0-ad11-15722c2e88d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:22.980937 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc17689b-58f7-4ea0-ad11-15722c2e88d1 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:22.991676 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50936c8-ee78-46db-8832-3780da431612 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:22.992598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50936c8-ee78-46db-8832-3780da431612 service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:22.994195 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50936c8-ee78-46db-8832-3780da431612 service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:22.994669 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50936c8-ee78-46db-8832-3780da431612 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:22.999158 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d50936c8-ee78-46db-8832-3780da431612 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:23.019660 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:23.024879 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50936c8-ee78-46db-8832-3780da431612 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:23.028180 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50936c8-ee78-46db-8832-3780da431612 service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:23.042727 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1288/2572] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:40:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:40:23.558346 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f0f0d-1e78-48b3-bce2-1aabfad6703d service cinder] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:23.629746 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f0f0d-1e78-48b3-bce2-1aabfad6703d service cinder] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:23.634220 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609a6f5-a5d0-4ca5-8de7-f44d5207894d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:23.634885 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609a6f5-a5d0-4ca5-8de7-f44d5207894d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:23.635161 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609a6f5-a5d0-4ca5-8de7-f44d5207894d admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:23.635434 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609a6f5-a5d0-4ca5-8de7-f44d5207894d admin admin] PATH_INFO: `/v3/users/1b570f2f0cf04c7e849ff6c55b33ca49` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:23.636544 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f609a6f5-a5d0-4ca5-8de7-f44d5207894d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b570f2f0cf04c7e849ff6c55b33ca49)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:23.638489 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f609a6f5-a5d0-4ca5-8de7-f44d5207894d admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:23.691676 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-f609a6f5-a5d0-4ca5-8de7-f44d5207894d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b570f2f0cf04c7e849ff6c55b33ca49'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:23.692394 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-f609a6f5-a5d0-4ca5-8de7-f44d5207894d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b570f2f0cf04c7e849ff6c55b33ca49'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:23.700454 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-f609a6f5-a5d0-4ca5-8de7-f44d5207894d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b570f2f0cf04c7e849ff6c55b33ca49'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:23.701227 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-f609a6f5-a5d0-4ca5-8de7-f44d5207894d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b570f2f0cf04c7e849ff6c55b33ca49'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:23.711233 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:23.717732 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-f609a6f5-a5d0-4ca5-8de7-f44d5207894d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b570f2f0cf04c7e849ff6c55b33ca49'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:23.735001 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1285/2573] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:40:23 2022] DELETE /identity/v3/users/1b570f2f0cf04c7e849ff6c55b33ca49 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:23.737488 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c953b-dab3-4cb3-ab19-708e801b78b3 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:23.737782 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c953b-dab3-4cb3-ab19-708e801b78b3 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:23.738058 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c953b-dab3-4cb3-ab19-708e801b78b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:23.758404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-558c953b-dab3-4cb3-ab19-708e801b78b3 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:23.759732 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-558c953b-dab3-4cb3-ab19-708e801b78b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:23.827407 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1289/2574] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:40:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:23.836941 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609a6f5-a5d0-4ca5-8de7-f44d5207894d admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:23.838536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-f609a6f5-a5d0-4ca5-8de7-f44d5207894d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:23.896318 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609a6f5-a5d0-4ca5-8de7-f44d5207894d admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:23.899951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46637810-69cd-4983-bd86-0918a6894d66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:23.900553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46637810-69cd-4983-bd86-0918a6894d66 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:23.900826 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46637810-69cd-4983-bd86-0918a6894d66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:23.901098 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46637810-69cd-4983-bd86-0918a6894d66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:23.910904 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46637810-69cd-4983-bd86-0918a6894d66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:23.912369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46637810-69cd-4983-bd86-0918a6894d66 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:23.921878 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1286/2575] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:40:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:40:24.188695 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c953b-dab3-4cb3-ab19-708e801b78b3 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:24.190186 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-558c953b-dab3-4cb3-ab19-708e801b78b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:24.209080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c953b-dab3-4cb3-ab19-708e801b78b3 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:24.213973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a563b5-fb90-4630-ae16-1de735230a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:24.214657 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a563b5-fb90-4630-ae16-1de735230a0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:24.214937 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a563b5-fb90-4630-ae16-1de735230a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:24.215209 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a563b5-fb90-4630-ae16-1de735230a0b admin admin] PATH_INFO: `/v3/projects/92f8156b0de74c0da84febb870b38e9b` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:24.216322 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a563b5-fb90-4630-ae16-1de735230a0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=92f8156b0de74c0da84febb870b38e9b)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:24.218218 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a563b5-fb90-4630-ae16-1de735230a0b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:24.264223 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f6a563b5-fb90-4630-ae16-1de735230a0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92f8156b0de74c0da84febb870b38e9b', 'request_id': 'req-f6a563b5-fb90-4630-ae16-1de735230a0b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:24.270172 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f6a563b5-fb90-4630-ae16-1de735230a0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92f8156b0de74c0da84febb870b38e9b', 'request_id': 'req-f6a563b5-fb90-4630-ae16-1de735230a0b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:24.277308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f6a563b5-fb90-4630-ae16-1de735230a0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92f8156b0de74c0da84febb870b38e9b', 'request_id': 'req-f6a563b5-fb90-4630-ae16-1de735230a0b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:24.287506 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f6a563b5-fb90-4630-ae16-1de735230a0b admin admin] The token cache is being invalidate because project 92f8156b0de74c0da84febb870b38e9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:40:24.287795 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f6a563b5-fb90-4630-ae16-1de735230a0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:24.288417 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f6a563b5-fb90-4630-ae16-1de735230a0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:24.289887 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1290/2576] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:40:24 2022] DELETE /identity/v3/projects/92f8156b0de74c0da84febb870b38e9b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:24.294558 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46637810-69cd-4983-bd86-0918a6894d66 service neutron] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:24.304151 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46637810-69cd-4983-bd86-0918a6894d66 service neutron] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:24.361706 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f63f2-08e2-4bec-a74b-dd70c07cd0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:24.362285 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f63f2-08e2-4bec-a74b-dd70c07cd0a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:24.362565 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f63f2-08e2-4bec-a74b-dd70c07cd0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:24.362839 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f63f2-08e2-4bec-a74b-dd70c07cd0a3 admin admin] PATH_INFO: `/v3/users/5765cf4825b44ec7a98105476d2b6227` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:24.363979 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f63f2-08e2-4bec-a74b-dd70c07cd0a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5765cf4825b44ec7a98105476d2b6227)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:24.365902 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f63f2-08e2-4bec-a74b-dd70c07cd0a3 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:24.414807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-803f63f2-08e2-4bec-a74b-dd70c07cd0a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5765cf4825b44ec7a98105476d2b6227'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:24.415469 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-803f63f2-08e2-4bec-a74b-dd70c07cd0a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5765cf4825b44ec7a98105476d2b6227'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:24.423739 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-803f63f2-08e2-4bec-a74b-dd70c07cd0a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5765cf4825b44ec7a98105476d2b6227'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:24.424355 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-803f63f2-08e2-4bec-a74b-dd70c07cd0a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5765cf4825b44ec7a98105476d2b6227'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:24.436971 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-803f63f2-08e2-4bec-a74b-dd70c07cd0a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5765cf4825b44ec7a98105476d2b6227'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:24.450467 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1287/2577] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:40:24 2022] DELETE /identity/v3/users/5765cf4825b44ec7a98105476d2b6227 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:24.720265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a563b5-fb90-4630-ae16-1de735230a0b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:24.721722 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f6a563b5-fb90-4630-ae16-1de735230a0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:24.735617 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a563b5-fb90-4630-ae16-1de735230a0b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:24.793553 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10616f37-839c-4742-88c0-f4cbe3cb5a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:24.794226 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10616f37-839c-4742-88c0-f4cbe3cb5a3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:24.794543 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10616f37-839c-4742-88c0-f4cbe3cb5a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:24.794820 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10616f37-839c-4742-88c0-f4cbe3cb5a3b admin admin] PATH_INFO: `/v3/projects/dcc97eabea5740669a89ca9f4c9f98a2` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:24.795954 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10616f37-839c-4742-88c0-f4cbe3cb5a3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcc97eabea5740669a89ca9f4c9f98a2)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:24.797821 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10616f37-839c-4742-88c0-f4cbe3cb5a3b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:24.818525 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:24.845042 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-10616f37-839c-4742-88c0-f4cbe3cb5a3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcc97eabea5740669a89ca9f4c9f98a2', 'request_id': 'req-10616f37-839c-4742-88c0-f4cbe3cb5a3b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:24.852107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-10616f37-839c-4742-88c0-f4cbe3cb5a3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcc97eabea5740669a89ca9f4c9f98a2', 'request_id': 'req-10616f37-839c-4742-88c0-f4cbe3cb5a3b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:24.860318 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-10616f37-839c-4742-88c0-f4cbe3cb5a3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcc97eabea5740669a89ca9f4c9f98a2', 'request_id': 'req-10616f37-839c-4742-88c0-f4cbe3cb5a3b'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:24.870775 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-10616f37-839c-4742-88c0-f4cbe3cb5a3b admin admin] The token cache is being invalidate because project dcc97eabea5740669a89ca9f4c9f98a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:40:24.871007 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-10616f37-839c-4742-88c0-f4cbe3cb5a3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:24.871628 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-10616f37-839c-4742-88c0-f4cbe3cb5a3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:24.873097 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1291/2578] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:40:24 2022] DELETE /identity/v3/projects/dcc97eabea5740669a89ca9f4c9f98a2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:24.878341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e0558-92c6-46b9-ba80-1fa96ccbb736 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:24.879071 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e0558-92c6-46b9-ba80-1fa96ccbb736 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:24.879071 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e0558-92c6-46b9-ba80-1fa96ccbb736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:24.887840 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:24.900455 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-c71e0558-92c6-46b9-ba80-1fa96ccbb736 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:24.902681 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-c71e0558-92c6-46b9-ba80-1fa96ccbb736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:24.962527 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1288/2579] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:40:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:24.971809 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620c3f7-9096-426f-aabd-e3c24ee5cb8d None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:24.972341 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620c3f7-9096-426f-aabd-e3c24ee5cb8d None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:24.972613 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620c3f7-9096-426f-aabd-e3c24ee5cb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:24.999189 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-f620c3f7-9096-426f-aabd-e3c24ee5cb8d None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:25.000584 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f620c3f7-9096-426f-aabd-e3c24ee5cb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:25.008853 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1292/2580] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:40:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:25.013011 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e0558-92c6-46b9-ba80-1fa96ccbb736 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:25.022652 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e0558-92c6-46b9-ba80-1fa96ccbb736 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:25.027676 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebdd86f-c439-4f1e-a144-10057cd5c847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:25.028275 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebdd86f-c439-4f1e-a144-10057cd5c847 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:25.028556 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebdd86f-c439-4f1e-a144-10057cd5c847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:25.028831 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebdd86f-c439-4f1e-a144-10057cd5c847 admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:25.029472 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebdd86f-c439-4f1e-a144-10057cd5c847 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:40:25.029774 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebdd86f-c439-4f1e-a144-10057cd5c847 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:25.031411 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebdd86f-c439-4f1e-a144-10057cd5c847 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:25.043642 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1289/2581] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:40:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:40:25.047864 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620c3f7-9096-426f-aabd-e3c24ee5cb8d None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:25.056970 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620c3f7-9096-426f-aabd-e3c24ee5cb8d None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:25.061633 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb4676-4217-43b4-8041-0321fdd677d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:25.062255 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb4676-4217-43b4-8041-0321fdd677d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:25.062542 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb4676-4217-43b4-8041-0321fdd677d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:25.062815 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb4676-4217-43b4-8041-0321fdd677d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:25.063451 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb4676-4217-43b4-8041-0321fdd677d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:25.065383 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb4676-4217-43b4-8041-0321fdd677d1 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:25.091992 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1293/2582] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:40:25 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:40:25.096264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebdd86f-c439-4f1e-a144-10057cd5c847 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:25.107004 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebdd86f-c439-4f1e-a144-10057cd5c847 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:25.163439 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01716eed-e83f-4d87-bb55-6067e942a964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:25.164056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01716eed-e83f-4d87-bb55-6067e942a964 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:25.164338 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01716eed-e83f-4d87-bb55-6067e942a964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:25.164611 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01716eed-e83f-4d87-bb55-6067e942a964 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:25.165248 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01716eed-e83f-4d87-bb55-6067e942a964 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:25.167235 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01716eed-e83f-4d87-bb55-6067e942a964 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:25.189067 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1290/2583] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:40:25 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:40:25.193148 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb4676-4217-43b4-8041-0321fdd677d1 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:25.203447 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb4676-4217-43b4-8041-0321fdd677d1 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:25.208038 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd2ee55-89b6-476c-85e4-a59cb46aad9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:25.208638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd2ee55-89b6-476c-85e4-a59cb46aad9b admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:25.208922 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd2ee55-89b6-476c-85e4-a59cb46aad9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:25.209197 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd2ee55-89b6-476c-85e4-a59cb46aad9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:25.209836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd2ee55-89b6-476c-85e4-a59cb46aad9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:25.211497 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd2ee55-89b6-476c-85e4-a59cb46aad9b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:25.221779 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1294/2584] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:40:25.225756 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01716eed-e83f-4d87-bb55-6067e942a964 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:25.236477 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01716eed-e83f-4d87-bb55-6067e942a964 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:25.241100 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df356d1-5753-44be-a132-866cdb965364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:25.241705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df356d1-5753-44be-a132-866cdb965364 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:25.241984 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df356d1-5753-44be-a132-866cdb965364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:25.242293 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df356d1-5753-44be-a132-866cdb965364 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:25.242940 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df356d1-5753-44be-a132-866cdb965364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:25.244535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df356d1-5753-44be-a132-866cdb965364 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:25.255412 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1291/2585] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:40:25.259754 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd2ee55-89b6-476c-85e4-a59cb46aad9b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:25.273075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd2ee55-89b6-476c-85e4-a59cb46aad9b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:25.281344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a1cb5b-fec0-4973-8dfc-2bcb0029950f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:25.283591 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1cb5b-fec0-4973-8dfc-2bcb0029950f admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:25.284067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1cb5b-fec0-4973-8dfc-2bcb0029950f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:25.284817 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1cb5b-fec0-4973-8dfc-2bcb0029950f admin admin] PATH_INFO: `/v3/projects/4c660e818bd942f3b52ef3c63a8f4e2e/users/50ea7ca27ba44cf38bc252093f94da54/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:25.314956 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a1cb5b-fec0-4973-8dfc-2bcb0029950f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c660e818bd942f3b52ef3c63a8f4e2e, user_id=50ea7ca27ba44cf38bc252093f94da54, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:25.325706 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a1cb5b-fec0-4973-8dfc-2bcb0029950f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:25.349206 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1295/2586] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:40:25 2022] PUT /identity/v3/projects/4c660e818bd942f3b52ef3c63a8f4e2e/users/50ea7ca27ba44cf38bc252093f94da54/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:25.359992 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb061bbb-cb59-4ec0-8cef-7d4ca21c47dc None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:25.360940 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb061bbb-cb59-4ec0-8cef-7d4ca21c47dc None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:25.361344 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb061bbb-cb59-4ec0-8cef-7d4ca21c47dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:25.392564 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-bb061bbb-cb59-4ec0-8cef-7d4ca21c47dc None None] MFA Rules not processed for user `50ea7ca27ba44cf38bc252093f94da54`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:25.393765 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-bb061bbb-cb59-4ec0-8cef-7d4ca21c47dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:25.454311 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:25.457982 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1292/2587] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:40:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:25.469500 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a1cb5b-fec0-4973-8dfc-2bcb0029950f admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:25.471159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-57a1cb5b-fec0-4973-8dfc-2bcb0029950f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:25.528109 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a1cb5b-fec0-4973-8dfc-2bcb0029950f admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:25.531741 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5656fb-e078-4006-a68d-aa91b40bb9eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:25.532337 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5656fb-e078-4006-a68d-aa91b40bb9eb service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:25.532614 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5656fb-e078-4006-a68d-aa91b40bb9eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:25.532902 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5656fb-e078-4006-a68d-aa91b40bb9eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:25.541007 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5656fb-e078-4006-a68d-aa91b40bb9eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:25.542491 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5656fb-e078-4006-a68d-aa91b40bb9eb service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:25.551416 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1296/2588] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:40:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:40:28.685702 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb061bbb-cb59-4ec0-8cef-7d4ca21c47dc None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:28.749352 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb061bbb-cb59-4ec0-8cef-7d4ca21c47dc None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:28.753848 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a501b0-f1fa-4316-a988-ed8413e8c645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:28.754508 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a501b0-f1fa-4316-a988-ed8413e8c645 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:28.754790 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a501b0-f1fa-4316-a988-ed8413e8c645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:28.755063 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a501b0-f1fa-4316-a988-ed8413e8c645 admin admin] PATH_INFO: `/v3/users/50ea7ca27ba44cf38bc252093f94da54` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:28.756182 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a501b0-f1fa-4316-a988-ed8413e8c645 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50ea7ca27ba44cf38bc252093f94da54)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:28.758103 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a501b0-f1fa-4316-a988-ed8413e8c645 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:28.805420 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-61a501b0-f1fa-4316-a988-ed8413e8c645 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50ea7ca27ba44cf38bc252093f94da54'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:28.806066 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-61a501b0-f1fa-4316-a988-ed8413e8c645 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50ea7ca27ba44cf38bc252093f94da54'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:28.815098 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-61a501b0-f1fa-4316-a988-ed8413e8c645 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50ea7ca27ba44cf38bc252093f94da54'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:28.815702 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-61a501b0-f1fa-4316-a988-ed8413e8c645 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50ea7ca27ba44cf38bc252093f94da54'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:28.829102 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-61a501b0-f1fa-4316-a988-ed8413e8c645 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50ea7ca27ba44cf38bc252093f94da54'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:28.841865 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1293/2589] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:40:28 2022] DELETE /identity/v3/users/50ea7ca27ba44cf38bc252093f94da54 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:28.846108 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55432c-b32c-4c39-99a4-4cc127bcb253 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:28.847030 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55432c-b32c-4c39-99a4-4cc127bcb253 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:28.847030 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55432c-b32c-4c39-99a4-4cc127bcb253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:28.865872 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-2f55432c-b32c-4c39-99a4-4cc127bcb253 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:28.867312 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2f55432c-b32c-4c39-99a4-4cc127bcb253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:28.893196 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f881bcb-f76f-4990-afcf-c637721cf228 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:28.893596 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f881bcb-f76f-4990-afcf-c637721cf228 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:28.893932 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f881bcb-f76f-4990-afcf-c637721cf228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:28.915836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:28.915998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-8f881bcb-f76f-4990-afcf-c637721cf228 None None] MFA Rules not processed for user `a5b3e53fd1314e17ad28c42a44e8eff3`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:28.917215 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-8f881bcb-f76f-4990-afcf-c637721cf228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:28.946159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1297/2590] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:40:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:28.956711 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55432c-b32c-4c39-99a4-4cc127bcb253 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:28.958078 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2f55432c-b32c-4c39-99a4-4cc127bcb253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:28.979858 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1294/2591] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Sep 1 16:40:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:40:28.992819 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f881bcb-f76f-4990-afcf-c637721cf228 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:28.994195 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-8f881bcb-f76f-4990-afcf-c637721cf228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:29.019491 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55432c-b32c-4c39-99a4-4cc127bcb253 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:29.022093 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:29.023663 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504e2350-47a7-4d4a-a14e-145697604e11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:29.024344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e2350-47a7-4d4a-a14e-145697604e11 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:29.024640 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e2350-47a7-4d4a-a14e-145697604e11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:29.024898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e2350-47a7-4d4a-a14e-145697604e11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:29.035129 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504e2350-47a7-4d4a-a14e-145697604e11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:29.036586 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504e2350-47a7-4d4a-a14e-145697604e11 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:29.046447 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1298/2592] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:40:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:40:29.068594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f881bcb-f76f-4990-afcf-c637721cf228 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:29.073615 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b7827-be95-498e-9c95-ba89787d121c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8137f6b2ced040bd9752239c7e987f03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:29.074731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b7827-be95-498e-9c95-ba89787d121c service swift] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:29.075011 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b7827-be95-498e-9c95-ba89787d121c service swift] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:29.075277 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b7827-be95-498e-9c95-ba89787d121c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:29.083842 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b7827-be95-498e-9c95-ba89787d121c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:29.085528 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b7827-be95-498e-9c95-ba89787d121c service swift] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:29.094465 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1295/2593] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 1 16:40:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 16:40:29.423850 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504e2350-47a7-4d4a-a14e-145697604e11 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:29.425565 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-504e2350-47a7-4d4a-a14e-145697604e11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:29.471354 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504e2350-47a7-4d4a-a14e-145697604e11 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:29.491515 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e3402-ea25-4a47-a6cd-f094f7712ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:29.493846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e3402-ea25-4a47-a6cd-f094f7712ee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:29.494469 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e3402-ea25-4a47-a6cd-f094f7712ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:29.494927 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e3402-ea25-4a47-a6cd-f094f7712ee1 admin admin] PATH_INFO: `/v3/projects/4c660e818bd942f3b52ef3c63a8f4e2e` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:29.500153 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51e3402-ea25-4a47-a6cd-f094f7712ee1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c660e818bd942f3b52ef3c63a8f4e2e)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:29.502434 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51e3402-ea25-4a47-a6cd-f094f7712ee1 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:29.584287 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e51e3402-ea25-4a47-a6cd-f094f7712ee1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c660e818bd942f3b52ef3c63a8f4e2e', 'request_id': 'req-e51e3402-ea25-4a47-a6cd-f094f7712ee1'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:29.591922 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e51e3402-ea25-4a47-a6cd-f094f7712ee1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c660e818bd942f3b52ef3c63a8f4e2e', 'request_id': 'req-e51e3402-ea25-4a47-a6cd-f094f7712ee1'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:29.602529 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e51e3402-ea25-4a47-a6cd-f094f7712ee1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c660e818bd942f3b52ef3c63a8f4e2e', 'request_id': 'req-e51e3402-ea25-4a47-a6cd-f094f7712ee1'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:29.621844 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e51e3402-ea25-4a47-a6cd-f094f7712ee1 admin admin] The token cache is being invalidate because project 4c660e818bd942f3b52ef3c63a8f4e2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:40:29.622456 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e51e3402-ea25-4a47-a6cd-f094f7712ee1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:29.623453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e51e3402-ea25-4a47-a6cd-f094f7712ee1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:29.626102 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1299/2594] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:40:29 2022] DELETE /identity/v3/projects/4c660e818bd942f3b52ef3c63a8f4e2e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:29.631687 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b69f0-a0a5-471d-8d97-1f3f9188aebb None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:29.632163 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b69f0-a0a5-471d-8d97-1f3f9188aebb None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:29.632618 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b69f0-a0a5-471d-8d97-1f3f9188aebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:29.657946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-f88b69f0-a0a5-471d-8d97-1f3f9188aebb None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:29.659257 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-f88b69f0-a0a5-471d-8d97-1f3f9188aebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:29.726319 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1296/2595] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:40:29.734223 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfd266-c429-4798-9756-79b677442122 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:29.734547 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfd266-c429-4798-9756-79b677442122 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:29.734835 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfd266-c429-4798-9756-79b677442122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:29.755759 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-dbbfd266-c429-4798-9756-79b677442122 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:29.757051 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-dbbfd266-c429-4798-9756-79b677442122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:29.766180 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1300/2596] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:40:29.770466 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b69f0-a0a5-471d-8d97-1f3f9188aebb None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:29.781259 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b69f0-a0a5-471d-8d97-1f3f9188aebb None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:29.786470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231cd77d-e6bb-4d8c-ad62-f98cb712ccac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:29.787541 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231cd77d-e6bb-4d8c-ad62-f98cb712ccac admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:29.788057 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231cd77d-e6bb-4d8c-ad62-f98cb712ccac admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:29.788565 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231cd77d-e6bb-4d8c-ad62-f98cb712ccac admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:29.789458 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231cd77d-e6bb-4d8c-ad62-f98cb712ccac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:40:29.789860 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231cd77d-e6bb-4d8c-ad62-f98cb712ccac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:29.791717 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231cd77d-e6bb-4d8c-ad62-f98cb712ccac admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:29.809911 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1297/2597] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:40:29.815464 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbfd266-c429-4798-9756-79b677442122 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:29.826192 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbfd266-c429-4798-9756-79b677442122 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:29.831135 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418af07-9941-4297-bb4b-3010d7c63017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:29.831749 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418af07-9941-4297-bb4b-3010d7c63017 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:29.832030 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418af07-9941-4297-bb4b-3010d7c63017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:29.832414 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418af07-9941-4297-bb4b-3010d7c63017 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:29.833136 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9418af07-9941-4297-bb4b-3010d7c63017 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:29.835293 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9418af07-9941-4297-bb4b-3010d7c63017 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:29.867629 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1301/2598] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:40:29 2022] POST /identity/v3/projects => generated 399 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:40:29.872936 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231cd77d-e6bb-4d8c-ad62-f98cb712ccac admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:29.884731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231cd77d-e6bb-4d8c-ad62-f98cb712ccac admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:29.945594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c00066-035a-439b-827e-a2a0b5bc83b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:29.946209 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c00066-035a-439b-827e-a2a0b5bc83b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:29.946523 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c00066-035a-439b-827e-a2a0b5bc83b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:29.946798 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c00066-035a-439b-827e-a2a0b5bc83b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:29.947500 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c00066-035a-439b-827e-a2a0b5bc83b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:29.949990 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c00066-035a-439b-827e-a2a0b5bc83b0 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:29.973747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1298/2599] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:40:29 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:40:29.977824 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418af07-9941-4297-bb4b-3010d7c63017 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:29.992048 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418af07-9941-4297-bb4b-3010d7c63017 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:29.998537 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b35a52-d954-46fd-90b1-4c425494b4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:29.999288 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b35a52-d954-46fd-90b1-4c425494b4c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:29.999606 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b35a52-d954-46fd-90b1-4c425494b4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:30.000075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b35a52-d954-46fd-90b1-4c425494b4c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:30.000729 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b35a52-d954-46fd-90b1-4c425494b4c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:30.002345 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b35a52-d954-46fd-90b1-4c425494b4c0 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:30.015001 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1302/2600] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:40:30.018832 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c00066-035a-439b-827e-a2a0b5bc83b0 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:30.029376 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c00066-035a-439b-827e-a2a0b5bc83b0 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:30.034126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe23703-9df0-4a0f-9b2f-77069c166bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:30.034767 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe23703-9df0-4a0f-9b2f-77069c166bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:30.035077 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe23703-9df0-4a0f-9b2f-77069c166bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:30.035311 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe23703-9df0-4a0f-9b2f-77069c166bb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:30.035963 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe23703-9df0-4a0f-9b2f-77069c166bb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:30.037566 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe23703-9df0-4a0f-9b2f-77069c166bb5 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:30.050788 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1299/2601] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:40:30.054932 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b35a52-d954-46fd-90b1-4c425494b4c0 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:30.066581 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b35a52-d954-46fd-90b1-4c425494b4c0 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:30.075778 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af63158-4249-42bd-933d-d71e5992a6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:30.076474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af63158-4249-42bd-933d-d71e5992a6e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:30.076760 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af63158-4249-42bd-933d-d71e5992a6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:30.077042 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af63158-4249-42bd-933d-d71e5992a6e4 admin admin] PATH_INFO: `/v3/projects/6c70bf6141a2494a9ace85bf851de6cc/users/5a186f9ddd4d42eea89a8cc2ce93d5da/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:30.093165 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af63158-4249-42bd-933d-d71e5992a6e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c70bf6141a2494a9ace85bf851de6cc, user_id=5a186f9ddd4d42eea89a8cc2ce93d5da, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:30.095780 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af63158-4249-42bd-933d-d71e5992a6e4 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:30.111136 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1303/2602] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:40:30 2022] PUT /identity/v3/projects/6c70bf6141a2494a9ace85bf851de6cc/users/5a186f9ddd4d42eea89a8cc2ce93d5da/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:30.121294 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dac58a-34e0-4b27-97ee-0c25c48b7764 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:30.121645 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dac58a-34e0-4b27-97ee-0c25c48b7764 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:30.121983 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dac58a-34e0-4b27-97ee-0c25c48b7764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:30.159531 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-25dac58a-34e0-4b27-97ee-0c25c48b7764 None None] MFA Rules not processed for user `5a186f9ddd4d42eea89a8cc2ce93d5da`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:30.160978 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-25dac58a-34e0-4b27-97ee-0c25c48b7764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:30.170746 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:30.230453 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1300/2603] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:40:30 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:30.242117 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af63158-4249-42bd-933d-d71e5992a6e4 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:30.243934 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7af63158-4249-42bd-933d-d71e5992a6e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:30.255455 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:30.309367 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af63158-4249-42bd-933d-d71e5992a6e4 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:30.312978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c512b-a54f-432d-b997-b2429d7c497d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:30.313586 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c512b-a54f-432d-b997-b2429d7c497d service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:30.313869 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c512b-a54f-432d-b997-b2429d7c497d service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:30.314142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c512b-a54f-432d-b997-b2429d7c497d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:30.322598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c512b-a54f-432d-b997-b2429d7c497d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:30.324776 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c512b-a54f-432d-b997-b2429d7c497d service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:30.334898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1304/2604] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:40:30 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:40:38.555307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dac58a-34e0-4b27-97ee-0c25c48b7764 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:38.620064 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dac58a-34e0-4b27-97ee-0c25c48b7764 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:38.626971 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd0890-9472-44e9-8ea0-3c933d68b633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:38.627764 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd0890-9472-44e9-8ea0-3c933d68b633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:38.627764 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd0890-9472-44e9-8ea0-3c933d68b633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:38.627764 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd0890-9472-44e9-8ea0-3c933d68b633 admin admin] PATH_INFO: `/v3/users/5a186f9ddd4d42eea89a8cc2ce93d5da` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:38.627764 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbd0890-9472-44e9-8ea0-3c933d68b633 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a186f9ddd4d42eea89a8cc2ce93d5da)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:38.627764 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbd0890-9472-44e9-8ea0-3c933d68b633 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:38.680081 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-1dbd0890-9472-44e9-8ea0-3c933d68b633 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a186f9ddd4d42eea89a8cc2ce93d5da'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:38.680762 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-1dbd0890-9472-44e9-8ea0-3c933d68b633 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a186f9ddd4d42eea89a8cc2ce93d5da'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:38.689171 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-1dbd0890-9472-44e9-8ea0-3c933d68b633 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a186f9ddd4d42eea89a8cc2ce93d5da'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:38.689816 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-1dbd0890-9472-44e9-8ea0-3c933d68b633 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a186f9ddd4d42eea89a8cc2ce93d5da'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:38.702660 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-1dbd0890-9472-44e9-8ea0-3c933d68b633 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a186f9ddd4d42eea89a8cc2ce93d5da'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:38.716372 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1301/2605] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:40:38 2022] DELETE /identity/v3/users/5a186f9ddd4d42eea89a8cc2ce93d5da => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:38.721282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408abe04-44c6-4c13-8a7e-0330de5ca82b None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:38.722171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408abe04-44c6-4c13-8a7e-0330de5ca82b None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:38.722171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408abe04-44c6-4c13-8a7e-0330de5ca82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:38.749557 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-408abe04-44c6-4c13-8a7e-0330de5ca82b None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:38.750845 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-408abe04-44c6-4c13-8a7e-0330de5ca82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:38.814293 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1305/2606] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:40:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:40:38.824408 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd0890-9472-44e9-8ea0-3c933d68b633 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:38.825492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-1dbd0890-9472-44e9-8ea0-3c933d68b633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:38.867089 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:38.907042 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd0890-9472-44e9-8ea0-3c933d68b633 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:38.910680 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb260c18-dd42-408f-a4bd-900e8975f648 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:38.911286 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb260c18-dd42-408f-a4bd-900e8975f648 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:38.911568 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb260c18-dd42-408f-a4bd-900e8975f648 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:38.911844 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb260c18-dd42-408f-a4bd-900e8975f648 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:38.920838 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb260c18-dd42-408f-a4bd-900e8975f648 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:38.922420 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb260c18-dd42-408f-a4bd-900e8975f648 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:38.932832 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1302/2607] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:40:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:40:39.217655 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408abe04-44c6-4c13-8a7e-0330de5ca82b None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:39.219112 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-408abe04-44c6-4c13-8a7e-0330de5ca82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:39.231346 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408abe04-44c6-4c13-8a7e-0330de5ca82b None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:39.235975 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2592d-e67e-4ea5-ab63-126d9b301ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:39.236684 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2592d-e67e-4ea5-ab63-126d9b301ff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:39.236969 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2592d-e67e-4ea5-ab63-126d9b301ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:39.237240 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2592d-e67e-4ea5-ab63-126d9b301ff3 admin admin] PATH_INFO: `/v3/projects/6c70bf6141a2494a9ace85bf851de6cc` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:39.238391 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde2592d-e67e-4ea5-ab63-126d9b301ff3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c70bf6141a2494a9ace85bf851de6cc)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:39.240320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde2592d-e67e-4ea5-ab63-126d9b301ff3 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:39.288880 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-fde2592d-e67e-4ea5-ab63-126d9b301ff3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c70bf6141a2494a9ace85bf851de6cc', 'request_id': 'req-fde2592d-e67e-4ea5-ab63-126d9b301ff3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:39.295011 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-fde2592d-e67e-4ea5-ab63-126d9b301ff3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c70bf6141a2494a9ace85bf851de6cc', 'request_id': 'req-fde2592d-e67e-4ea5-ab63-126d9b301ff3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:39.302191 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-fde2592d-e67e-4ea5-ab63-126d9b301ff3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c70bf6141a2494a9ace85bf851de6cc', 'request_id': 'req-fde2592d-e67e-4ea5-ab63-126d9b301ff3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:39.316127 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-fde2592d-e67e-4ea5-ab63-126d9b301ff3 admin admin] The token cache is being invalidate because project 6c70bf6141a2494a9ace85bf851de6cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:40:39.316500 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-fde2592d-e67e-4ea5-ab63-126d9b301ff3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:39.317227 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-fde2592d-e67e-4ea5-ab63-126d9b301ff3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:39.318886 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1306/2608] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:40:39 2022] DELETE /identity/v3/projects/6c70bf6141a2494a9ace85bf851de6cc => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:39.324377 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a201bc4-88aa-44be-b32d-32693ef6e57b None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:39.324763 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a201bc4-88aa-44be-b32d-32693ef6e57b None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:39.325105 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a201bc4-88aa-44be-b32d-32693ef6e57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:39.345874 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-6a201bc4-88aa-44be-b32d-32693ef6e57b None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:39.347498 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-6a201bc4-88aa-44be-b32d-32693ef6e57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:39.412871 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1303/2609] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:40:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:39.420717 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb4b50-6e73-4005-a1df-e78156c9da46 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:39.421103 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb4b50-6e73-4005-a1df-e78156c9da46 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:39.421597 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb4b50-6e73-4005-a1df-e78156c9da46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:39.443716 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-2adb4b50-6e73-4005-a1df-e78156c9da46 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:39.445277 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2adb4b50-6e73-4005-a1df-e78156c9da46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:39.454099 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1307/2610] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:40:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:39.458142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a201bc4-88aa-44be-b32d-32693ef6e57b None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:39.468515 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a201bc4-88aa-44be-b32d-32693ef6e57b None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:39.473354 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d8198-0024-433a-96c2-86d9ea3ab28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:39.473951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d8198-0024-433a-96c2-86d9ea3ab28b admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:39.474247 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d8198-0024-433a-96c2-86d9ea3ab28b admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:39.474543 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d8198-0024-433a-96c2-86d9ea3ab28b admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:39.475221 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d8198-0024-433a-96c2-86d9ea3ab28b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:40:39.475526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d8198-0024-433a-96c2-86d9ea3ab28b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:39.477984 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d8198-0024-433a-96c2-86d9ea3ab28b admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:39.491196 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1304/2611] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:40:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:40:39.496160 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb4b50-6e73-4005-a1df-e78156c9da46 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:39.505229 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb4b50-6e73-4005-a1df-e78156c9da46 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:39.509945 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc8127-dc63-45e2-ba82-a1a6975021c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:39.510576 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc8127-dc63-45e2-ba82-a1a6975021c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:39.510916 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc8127-dc63-45e2-ba82-a1a6975021c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:39.511774 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc8127-dc63-45e2-ba82-a1a6975021c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:39.512438 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fc8127-dc63-45e2-ba82-a1a6975021c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:39.514406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fc8127-dc63-45e2-ba82-a1a6975021c9 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:39.541175 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1308/2612] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:40:39 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:40:39.545236 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d8198-0024-433a-96c2-86d9ea3ab28b admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:39.555541 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d8198-0024-433a-96c2-86d9ea3ab28b admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:39.611869 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf3dc0-e4be-4c67-b013-c0162a7225e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:39.612472 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf3dc0-e4be-4c67-b013-c0162a7225e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:39.612757 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf3dc0-e4be-4c67-b013-c0162a7225e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:39.613032 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf3dc0-e4be-4c67-b013-c0162a7225e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:39.613700 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf3dc0-e4be-4c67-b013-c0162a7225e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:39.615689 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf3dc0-e4be-4c67-b013-c0162a7225e7 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:39.637546 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1305/2613] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:40:39 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:40:39.641844 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc8127-dc63-45e2-ba82-a1a6975021c9 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:39.653255 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc8127-dc63-45e2-ba82-a1a6975021c9 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:39.657836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f10b1-947c-471e-8e04-4ecd1b3f252c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:39.658466 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f10b1-947c-471e-8e04-4ecd1b3f252c admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:39.658778 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f10b1-947c-471e-8e04-4ecd1b3f252c admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:39.659052 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f10b1-947c-471e-8e04-4ecd1b3f252c admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:39.659683 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f10b1-947c-471e-8e04-4ecd1b3f252c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:39.661263 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f10b1-947c-471e-8e04-4ecd1b3f252c admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:39.662586 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:39.794076 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1309/2614] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:40:39.797908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf3dc0-e4be-4c67-b013-c0162a7225e7 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:39.809618 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf3dc0-e4be-4c67-b013-c0162a7225e7 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:39.814178 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f1667a-acca-4082-bb13-3202c1e3df5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:39.814809 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1667a-acca-4082-bb13-3202c1e3df5e admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:39.815094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1667a-acca-4082-bb13-3202c1e3df5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:39.815367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1667a-acca-4082-bb13-3202c1e3df5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:39.816007 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f1667a-acca-4082-bb13-3202c1e3df5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:39.817601 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f1667a-acca-4082-bb13-3202c1e3df5e admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:39.829560 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1306/2615] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:40:39.833614 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f10b1-947c-471e-8e04-4ecd1b3f252c admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:39.843677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f10b1-947c-471e-8e04-4ecd1b3f252c admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:39.848596 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f873ce-06ba-42bb-be15-75def8f0f03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:39.849237 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f873ce-06ba-42bb-be15-75def8f0f03a admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:39.849518 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f873ce-06ba-42bb-be15-75def8f0f03a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:39.849791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f873ce-06ba-42bb-be15-75def8f0f03a admin admin] PATH_INFO: `/v3/projects/13ab20a85384485e95e2c0dadcc8b43c/users/003d7b8eee13419780bb5e055a01bb77/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:39.864528 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f873ce-06ba-42bb-be15-75def8f0f03a admin admin] RBAC: Authorizing `identity:create_grant(project_id=13ab20a85384485e95e2c0dadcc8b43c, user_id=003d7b8eee13419780bb5e055a01bb77, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:39.867114 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f873ce-06ba-42bb-be15-75def8f0f03a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:39.880538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1310/2616] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:40:39 2022] PUT /identity/v3/projects/13ab20a85384485e95e2c0dadcc8b43c/users/003d7b8eee13419780bb5e055a01bb77/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:39.886035 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a52f87-3e80-45d9-a9b7-5920619003dc None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:39.886367 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a52f87-3e80-45d9-a9b7-5920619003dc None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:39.886647 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a52f87-3e80-45d9-a9b7-5920619003dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:39.905428 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-57a52f87-3e80-45d9-a9b7-5920619003dc None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:39.906671 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-57a52f87-3e80-45d9-a9b7-5920619003dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:39.941042 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:39.968692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1307/2617] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:40:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:39.978511 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f873ce-06ba-42bb-be15-75def8f0f03a admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:39.980732 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-e1f873ce-06ba-42bb-be15-75def8f0f03a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:40.037727 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f873ce-06ba-42bb-be15-75def8f0f03a admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:40.041356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb33742-3a9e-45ab-8f04-ea6aa6f472b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:40.041956 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb33742-3a9e-45ab-8f04-ea6aa6f472b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:40.042889 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb33742-3a9e-45ab-8f04-ea6aa6f472b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:40.043167 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb33742-3a9e-45ab-8f04-ea6aa6f472b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:40.051901 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb33742-3a9e-45ab-8f04-ea6aa6f472b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:40.053316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb33742-3a9e-45ab-8f04-ea6aa6f472b6 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:40.062579 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1311/2618] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:40:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:40:44.387058 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28505cfe-d750-45da-adae-77049b74a25b None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:44.387587 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28505cfe-d750-45da-adae-77049b74a25b None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:44.387587 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28505cfe-d750-45da-adae-77049b74a25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:44.417559 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-28505cfe-d750-45da-adae-77049b74a25b None None] MFA Rules not processed for user `003d7b8eee13419780bb5e055a01bb77`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:44.418840 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-28505cfe-d750-45da-adae-77049b74a25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:44.485238 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1308/2619] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:40:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:44.489938 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb33742-3a9e-45ab-8f04-ea6aa6f472b6 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:44.500967 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb33742-3a9e-45ab-8f04-ea6aa6f472b6 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:44.505764 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49dd71-b14c-49b9-b066-13d7eb015394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:44.506393 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49dd71-b14c-49b9-b066-13d7eb015394 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:44.506674 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49dd71-b14c-49b9-b066-13d7eb015394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:44.506944 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49dd71-b14c-49b9-b066-13d7eb015394 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:44.507578 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f49dd71-b14c-49b9-b066-13d7eb015394 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:44.509457 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f49dd71-b14c-49b9-b066-13d7eb015394 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:44.537798 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1312/2620] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:40:44 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:40:44.542873 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28505cfe-d750-45da-adae-77049b74a25b None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:44.553609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28505cfe-d750-45da-adae-77049b74a25b None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:44.612610 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c41062d-b05e-48ef-a00c-7718bba495a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:44.613216 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41062d-b05e-48ef-a00c-7718bba495a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:44.613601 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41062d-b05e-48ef-a00c-7718bba495a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:44.613802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41062d-b05e-48ef-a00c-7718bba495a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:44.614402 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c41062d-b05e-48ef-a00c-7718bba495a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:44.616323 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c41062d-b05e-48ef-a00c-7718bba495a5 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:44.640585 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1309/2621] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:40:44 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:40:44.641911 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49dd71-b14c-49b9-b066-13d7eb015394 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:44.652726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49dd71-b14c-49b9-b066-13d7eb015394 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:44.657299 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4aa89d-d1f6-4cfb-96a4-73846cc6406e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:44.657909 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4aa89d-d1f6-4cfb-96a4-73846cc6406e admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:44.658192 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4aa89d-d1f6-4cfb-96a4-73846cc6406e admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:44.658497 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4aa89d-d1f6-4cfb-96a4-73846cc6406e admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:44.659138 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4aa89d-d1f6-4cfb-96a4-73846cc6406e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:44.660714 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4aa89d-d1f6-4cfb-96a4-73846cc6406e admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:44.672333 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1313/2622] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:40:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:40:44.676436 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c41062d-b05e-48ef-a00c-7718bba495a5 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:44.687040 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c41062d-b05e-48ef-a00c-7718bba495a5 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:44.695030 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada9914-b0fe-45e5-b3a9-6a1a5c18cc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:44.695709 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada9914-b0fe-45e5-b3a9-6a1a5c18cc04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:44.695988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada9914-b0fe-45e5-b3a9-6a1a5c18cc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:44.696259 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada9914-b0fe-45e5-b3a9-6a1a5c18cc04 admin admin] PATH_INFO: `/v3/projects/4cf6935593374d03a401512b9f31df9b/users/f6dd4e06e49a429fbef5601af067ce20/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:44.715038 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ada9914-b0fe-45e5-b3a9-6a1a5c18cc04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cf6935593374d03a401512b9f31df9b, user_id=f6dd4e06e49a429fbef5601af067ce20, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:44.717928 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ada9914-b0fe-45e5-b3a9-6a1a5c18cc04 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:44.733383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1310/2623] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:40:44 2022] PUT /identity/v3/projects/4cf6935593374d03a401512b9f31df9b/users/f6dd4e06e49a429fbef5601af067ce20/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:44.737403 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4aa89d-d1f6-4cfb-96a4-73846cc6406e admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:44.748494 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4aa89d-d1f6-4cfb-96a4-73846cc6406e admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:44.891149 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2120dc-f3b1-44b0-9132-43c6dda75a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:44.893081 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2120dc-f3b1-44b0-9132-43c6dda75a43 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:44.894025 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2120dc-f3b1-44b0-9132-43c6dda75a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:44.894419 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2120dc-f3b1-44b0-9132-43c6dda75a43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:44.895184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2120dc-f3b1-44b0-9132-43c6dda75a43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:44.906901 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2120dc-f3b1-44b0-9132-43c6dda75a43 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:44.923607 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1314/2624] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:40:44 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:40:44.935414 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada9914-b0fe-45e5-b3a9-6a1a5c18cc04 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:44.953008 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada9914-b0fe-45e5-b3a9-6a1a5c18cc04 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:44.957992 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2b2b0-69b9-4654-87a8-12e4e27b7f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:44.958695 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2b2b0-69b9-4654-87a8-12e4e27b7f71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:44.959126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2b2b0-69b9-4654-87a8-12e4e27b7f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:44.959416 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2b2b0-69b9-4654-87a8-12e4e27b7f71 admin admin] PATH_INFO: `/v3/projects/4cf6935593374d03a401512b9f31df9b/users/f6dd4e06e49a429fbef5601af067ce20/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:44.961357 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2b2b0-69b9-4654-87a8-12e4e27b7f71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cf6935593374d03a401512b9f31df9b, user_id=f6dd4e06e49a429fbef5601af067ce20, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:44.976390 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2b2b0-69b9-4654-87a8-12e4e27b7f71 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:44.995823 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1311/2625] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:40:44 2022] PUT /identity/v3/projects/4cf6935593374d03a401512b9f31df9b/users/f6dd4e06e49a429fbef5601af067ce20/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:47.467541 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f880bdd-6d97-439e-aa27-1989a4ea3448 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:47.468422 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f880bdd-6d97-439e-aa27-1989a4ea3448 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:47.468924 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f880bdd-6d97-439e-aa27-1989a4ea3448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:47.489249 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-3f880bdd-6d97-439e-aa27-1989a4ea3448 None None] MFA Rules not processed for user `a5b3e53fd1314e17ad28c42a44e8eff3`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:47.491404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3f880bdd-6d97-439e-aa27-1989a4ea3448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:47.529816 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:47.562712 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1315/2626] 10.222.0.18 () {58 vars in 967 bytes} [Thu Sep 1 16:40:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:47.578508 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2b2b0-69b9-4654-87a8-12e4e27b7f71 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:47.580293 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-c2e2b2b0-69b9-4654-87a8-12e4e27b7f71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:47.634141 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:47.724304 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2b2b0-69b9-4654-87a8-12e4e27b7f71 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:47.732401 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5649578-1cdf-4c28-96b9-95052e4e5bc4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8137f6b2ced040bd9752239c7e987f03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:47.733311 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5649578-1cdf-4c28-96b9-95052e4e5bc4 service swift] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:47.733767 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5649578-1cdf-4c28-96b9-95052e4e5bc4 service swift] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:47.734196 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5649578-1cdf-4c28-96b9-95052e4e5bc4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:47.747147 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5649578-1cdf-4c28-96b9-95052e4e5bc4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:47.749256 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5649578-1cdf-4c28-96b9-95052e4e5bc4 service swift] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:47.782028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1312/2627] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 1 16:40:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:40:50.414358 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc199a6-425e-4c8c-b15e-5ec8e84bf9c3 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:50.414887 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc199a6-425e-4c8c-b15e-5ec8e84bf9c3 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:50.415563 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc199a6-425e-4c8c-b15e-5ec8e84bf9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:50.450698 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-9fc199a6-425e-4c8c-b15e-5ec8e84bf9c3 None None] MFA Rules not processed for user `f6dd4e06e49a429fbef5601af067ce20`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:50.451856 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9fc199a6-425e-4c8c-b15e-5ec8e84bf9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:50.533371 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1316/2628] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:40:50 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:50.545869 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5649578-1cdf-4c28-96b9-95052e4e5bc4 service swift] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:50.547415 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-a5649578-1cdf-4c28-96b9-95052e4e5bc4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:50.615128 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5649578-1cdf-4c28-96b9-95052e4e5bc4 service swift] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:50.620400 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db419d10-f3fd-4aff-b97b-38b515c60c17 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b09f6d550c9142bf956b225bf5172e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:50.621195 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db419d10-f3fd-4aff-b97b-38b515c60c17 service glance] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:50.622011 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db419d10-f3fd-4aff-b97b-38b515c60c17 service glance] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:50.622011 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db419d10-f3fd-4aff-b97b-38b515c60c17 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:50.695219 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db419d10-f3fd-4aff-b97b-38b515c60c17 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:50.696784 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db419d10-f3fd-4aff-b97b-38b515c60c17 service glance] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:50.707815 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1313/2629] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:40:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:40:50.844066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c502e1-4eff-4d45-8307-24e025221202 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:50.844360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c502e1-4eff-4d45-8307-24e025221202 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:50.844640 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c502e1-4eff-4d45-8307-24e025221202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:50.874650 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-d6c502e1-4eff-4d45-8307-24e025221202 None None] MFA Rules not processed for user `a5b3e53fd1314e17ad28c42a44e8eff3`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:50.875870 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d6c502e1-4eff-4d45-8307-24e025221202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:50.883367 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1317/2630] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Sep 1 16:40:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:40:50.897153 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db419d10-f3fd-4aff-b97b-38b515c60c17 service glance] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:50.911042 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db419d10-f3fd-4aff-b97b-38b515c60c17 service glance] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:50.916355 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7821fc1b-2730-4765-9ad3-7e940c3e36d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8137f6b2ced040bd9752239c7e987f03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:50.917505 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821fc1b-2730-4765-9ad3-7e940c3e36d2 service swift] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:50.917789 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821fc1b-2730-4765-9ad3-7e940c3e36d2 service swift] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:50.918063 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821fc1b-2730-4765-9ad3-7e940c3e36d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:50.927362 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7821fc1b-2730-4765-9ad3-7e940c3e36d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:50.930549 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7821fc1b-2730-4765-9ad3-7e940c3e36d2 service swift] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:50.941085 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1314/2631] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 1 16:40:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:40:58.245835 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c502e1-4eff-4d45-8307-24e025221202 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:58.319150 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c502e1-4eff-4d45-8307-24e025221202 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:58.322865 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1640841-dd26-46b0-bb79-f46e2236bcb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:58.324201 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1640841-dd26-46b0-bb79-f46e2236bcb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:58.324201 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1640841-dd26-46b0-bb79-f46e2236bcb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:58.324201 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1640841-dd26-46b0-bb79-f46e2236bcb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:58.326024 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a1640841-dd26-46b0-bb79-f46e2236bcb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:58.411245 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1640841-dd26-46b0-bb79-f46e2236bcb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:58.412600 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1640841-dd26-46b0-bb79-f46e2236bcb1 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:58.424822 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1318/2632] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:40:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:00.683896 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ba4c8-3587-4e5a-a963-9d3626a4754e None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:00.684824 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ba4c8-3587-4e5a-a963-9d3626a4754e None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:00.685114 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ba4c8-3587-4e5a-a963-9d3626a4754e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:00.705875 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-018ba4c8-3587-4e5a-a963-9d3626a4754e None None] MFA Rules not processed for user `003d7b8eee13419780bb5e055a01bb77`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:00.707185 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-018ba4c8-3587-4e5a-a963-9d3626a4754e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:00.714765 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1315/2633] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:02.092427 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd3d85-a356-4225-aafb-649005343833 None None] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:02.092948 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd3d85-a356-4225-aafb-649005343833 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:02.093129 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd3d85-a356-4225-aafb-649005343833 None None] PATH_INFO: `/` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:02.095031 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1319/2634] 10.222.0.18 () {54 vars in 961 bytes} [Thu Sep 1 16:41:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:41:02.106088 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb039a8-a90c-47ac-9ed6-9b6aa0ebf4a3 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:02.106638 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb039a8-a90c-47ac-9ed6-9b6aa0ebf4a3 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:02.106723 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb039a8-a90c-47ac-9ed6-9b6aa0ebf4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:02.141569 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-5fb039a8-a90c-47ac-9ed6-9b6aa0ebf4a3 None None] MFA Rules not processed for user `3ddfb151bbf844efae73ec1990e7eaa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:02.142877 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-5fb039a8-a90c-47ac-9ed6-9b6aa0ebf4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:02.170470 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd3d85-a356-4225-aafb-649005343833 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:02.172636 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7fdd3d85-a356-4225-aafb-649005343833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:02.191028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd3d85-a356-4225-aafb-649005343833 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:02.196567 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f04830-684e-435a-acae-4ab2ec0668cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:02.197219 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f04830-684e-435a-acae-4ab2ec0668cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:02.197543 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f04830-684e-435a-acae-4ab2ec0668cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:02.197814 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f04830-684e-435a-acae-4ab2ec0668cb admin admin] PATH_INFO: `/v3/users/73624e79ccdd4782beddf663bb6d02e1` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:02.199039 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f04830-684e-435a-acae-4ab2ec0668cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=73624e79ccdd4782beddf663bb6d02e1)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:02.201447 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f04830-684e-435a-acae-4ab2ec0668cb admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:02.206724 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:02.209444 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:02.235704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1316/2635] 10.222.0.18 () {58 vars in 1058 bytes} [Thu Sep 1 16:41:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:02.244031 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb039a8-a90c-47ac-9ed6-9b6aa0ebf4a3 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:02.255755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb039a8-a90c-47ac-9ed6-9b6aa0ebf4a3 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:02.256614 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-43f04830-684e-435a-acae-4ab2ec0668cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73624e79ccdd4782beddf663bb6d02e1'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:02.261133 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8198251-267b-4f03-a184-f8b2397f2dd0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ddfb151bbf844efae73ec1990e7eaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:02.262147 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8198251-267b-4f03-a184-f8b2397f2dd0 service placement] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:02.262625 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8198251-267b-4f03-a184-f8b2397f2dd0 service placement] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:02.262908 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8198251-267b-4f03-a184-f8b2397f2dd0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:02.265147 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-d8198251-267b-4f03-a184-f8b2397f2dd0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:02.265950 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-43f04830-684e-435a-acae-4ab2ec0668cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73624e79ccdd4782beddf663bb6d02e1'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:02.266686 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-43f04830-684e-435a-acae-4ab2ec0668cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73624e79ccdd4782beddf663bb6d02e1'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:02.278876 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8198251-267b-4f03-a184-f8b2397f2dd0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:02.280524 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8198251-267b-4f03-a184-f8b2397f2dd0 service placement] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:02.291729 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-43f04830-684e-435a-acae-4ab2ec0668cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73624e79ccdd4782beddf663bb6d02e1'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:02.292449 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-43f04830-684e-435a-acae-4ab2ec0668cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73624e79ccdd4782beddf663bb6d02e1'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:02.294741 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1317/2636] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:41:02.311234 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1320/2637] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:41:02 2022] DELETE /identity/v3/users/73624e79ccdd4782beddf663bb6d02e1 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:41:02.498628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8198251-267b-4f03-a184-f8b2397f2dd0 service placement] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:02.509593 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8198251-267b-4f03-a184-f8b2397f2dd0 service placement] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:02.566961 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3e92b-d74c-4b3d-80da-17838661755d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:02.567594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3e92b-d74c-4b3d-80da-17838661755d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:02.567877 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3e92b-d74c-4b3d-80da-17838661755d admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:02.568151 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3e92b-d74c-4b3d-80da-17838661755d admin admin] PATH_INFO: `/v3/projects/aab96a55de014d8cac2c004ba7cadedd` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:02.569262 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d3e92b-d74c-4b3d-80da-17838661755d admin admin] RBAC: Authorizing `identity:delete_project(project_id=aab96a55de014d8cac2c004ba7cadedd)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:02.571227 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d3e92b-d74c-4b3d-80da-17838661755d admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:02.617957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c6d3e92b-d74c-4b3d-80da-17838661755d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aab96a55de014d8cac2c004ba7cadedd', 'request_id': 'req-c6d3e92b-d74c-4b3d-80da-17838661755d'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:02.625110 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c6d3e92b-d74c-4b3d-80da-17838661755d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aab96a55de014d8cac2c004ba7cadedd', 'request_id': 'req-c6d3e92b-d74c-4b3d-80da-17838661755d'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:02.631295 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c6d3e92b-d74c-4b3d-80da-17838661755d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aab96a55de014d8cac2c004ba7cadedd', 'request_id': 'req-c6d3e92b-d74c-4b3d-80da-17838661755d'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:02.641510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c6d3e92b-d74c-4b3d-80da-17838661755d admin admin] The token cache is being invalidate because project aab96a55de014d8cac2c004ba7cadedd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:41:02.641650 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c6d3e92b-d74c-4b3d-80da-17838661755d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:02.642345 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c6d3e92b-d74c-4b3d-80da-17838661755d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:02.644924 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1318/2638] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:41:02 2022] DELETE /identity/v3/projects/aab96a55de014d8cac2c004ba7cadedd => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:02.649318 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3da2c-954f-4444-9217-58830443273b None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:02.649615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3da2c-954f-4444-9217-58830443273b None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:02.649891 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3da2c-954f-4444-9217-58830443273b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:02.670269 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-c4f3da2c-954f-4444-9217-58830443273b None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:02.671467 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c4f3da2c-954f-4444-9217-58830443273b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:02.734907 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1321/2639] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:02.742744 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfebf81-b0ba-49a6-bd58-ad9497cfa37f None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:02.743028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfebf81-b0ba-49a6-bd58-ad9497cfa37f None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:02.743304 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfebf81-b0ba-49a6-bd58-ad9497cfa37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:02.764053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-fbfebf81-b0ba-49a6-bd58-ad9497cfa37f None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:02.765233 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-fbfebf81-b0ba-49a6-bd58-ad9497cfa37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:02.772857 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1319/2640] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:41:02.777140 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3da2c-954f-4444-9217-58830443273b None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:02.786579 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3da2c-954f-4444-9217-58830443273b None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:02.791061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58460c32-bd73-47d5-ac12-8d44eacabf00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:02.791698 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58460c32-bd73-47d5-ac12-8d44eacabf00 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:02.791977 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58460c32-bd73-47d5-ac12-8d44eacabf00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:02.792248 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58460c32-bd73-47d5-ac12-8d44eacabf00 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:02.792889 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58460c32-bd73-47d5-ac12-8d44eacabf00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:41:02.793187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58460c32-bd73-47d5-ac12-8d44eacabf00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:02.794813 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58460c32-bd73-47d5-ac12-8d44eacabf00 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:02.806333 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1322/2641] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:41:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:41:02.811114 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfebf81-b0ba-49a6-bd58-ad9497cfa37f None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:02.820519 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfebf81-b0ba-49a6-bd58-ad9497cfa37f None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:02.826132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af69e5b-b756-4ef0-b0c5-8553e7ea9ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:02.826789 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af69e5b-b756-4ef0-b0c5-8553e7ea9ca7 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:02.827099 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af69e5b-b756-4ef0-b0c5-8553e7ea9ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:02.827373 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af69e5b-b756-4ef0-b0c5-8553e7ea9ca7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:02.828013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af69e5b-b756-4ef0-b0c5-8553e7ea9ca7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:02.829931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af69e5b-b756-4ef0-b0c5-8553e7ea9ca7 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:02.870176 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1320/2642] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:41:02 2022] POST /identity/v3/projects => generated 367 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:02.873942 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58460c32-bd73-47d5-ac12-8d44eacabf00 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:02.885008 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58460c32-bd73-47d5-ac12-8d44eacabf00 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:02.954006 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d8e19-ae82-42fb-9aa6-57314cb15e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:02.954997 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d8e19-ae82-42fb-9aa6-57314cb15e16 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:02.955539 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d8e19-ae82-42fb-9aa6-57314cb15e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:02.956024 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d8e19-ae82-42fb-9aa6-57314cb15e16 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:02.956868 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16d8e19-ae82-42fb-9aa6-57314cb15e16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:02.959561 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16d8e19-ae82-42fb-9aa6-57314cb15e16 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:02.982616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:02.995682 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1323/2643] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:41:02 2022] POST /identity/v3/users => generated 284 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:02.995889 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af69e5b-b756-4ef0-b0c5-8553e7ea9ca7 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:03.007579 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af69e5b-b756-4ef0-b0c5-8553e7ea9ca7 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:03.012601 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c8ffcd-b51f-40f0-816a-b81b68cfcda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:03.013246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8ffcd-b51f-40f0-816a-b81b68cfcda1 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:03.013552 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8ffcd-b51f-40f0-816a-b81b68cfcda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:03.013813 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8ffcd-b51f-40f0-816a-b81b68cfcda1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:03.014513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c8ffcd-b51f-40f0-816a-b81b68cfcda1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:03.016166 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c8ffcd-b51f-40f0-816a-b81b68cfcda1 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:03.026743 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1321/2644] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:41:03.030808 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d8e19-ae82-42fb-9aa6-57314cb15e16 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:03.042262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d8e19-ae82-42fb-9aa6-57314cb15e16 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:03.048553 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb4724-bfa5-4f95-a54e-e3855399a31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:03.049163 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb4724-bfa5-4f95-a54e-e3855399a31e admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:03.049443 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb4724-bfa5-4f95-a54e-e3855399a31e admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:03.049717 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb4724-bfa5-4f95-a54e-e3855399a31e admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:03.050402 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb4724-bfa5-4f95-a54e-e3855399a31e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:03.051971 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb4724-bfa5-4f95-a54e-e3855399a31e admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:03.064661 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1324/2645] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:41:03.069277 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c8ffcd-b51f-40f0-816a-b81b68cfcda1 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:03.080949 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c8ffcd-b51f-40f0-816a-b81b68cfcda1 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:03.085637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b076ad45-fba9-436e-ba34-2373ae80218d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:03.086354 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076ad45-fba9-436e-ba34-2373ae80218d admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:03.086639 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076ad45-fba9-436e-ba34-2373ae80218d admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:03.086916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076ad45-fba9-436e-ba34-2373ae80218d admin admin] PATH_INFO: `/v3/projects/63a01bf1c1294eb793db2c2cce4df3f4/users/087f321f6997400b847fe1263af9f89e/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:03.103195 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b076ad45-fba9-436e-ba34-2373ae80218d admin admin] RBAC: Authorizing `identity:create_grant(project_id=63a01bf1c1294eb793db2c2cce4df3f4, user_id=087f321f6997400b847fe1263af9f89e, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:03.107560 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b076ad45-fba9-436e-ba34-2373ae80218d admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:03.109966 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:03.124168 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1322/2646] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:41:03 2022] PUT /identity/v3/projects/63a01bf1c1294eb793db2c2cce4df3f4/users/087f321f6997400b847fe1263af9f89e/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:03.136230 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532f29b-7dd4-4b04-bf33-104887d90ed3 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:03.136750 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532f29b-7dd4-4b04-bf33-104887d90ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:03.136869 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532f29b-7dd4-4b04-bf33-104887d90ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:03.164791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-f532f29b-7dd4-4b04-bf33-104887d90ed3 None None] MFA Rules not processed for user `087f321f6997400b847fe1263af9f89e`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:03.166018 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f532f29b-7dd4-4b04-bf33-104887d90ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:03.233559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1325/2647] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:03.239702 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b076ad45-fba9-436e-ba34-2373ae80218d admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:03.281795 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50508a21-6217-42b7-a603-20efbe90e00b None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:03.282201 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50508a21-6217-42b7-a603-20efbe90e00b None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:03.282600 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50508a21-6217-42b7-a603-20efbe90e00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:03.304668 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-50508a21-6217-42b7-a603-20efbe90e00b None None] MFA Rules not processed for user `a5b3e53fd1314e17ad28c42a44e8eff3`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:03.305920 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-50508a21-6217-42b7-a603-20efbe90e00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:03.310027 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b076ad45-fba9-436e-ba34-2373ae80218d admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:03.314968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ecca94-cece-44c2-a2d3-8812be9ab675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:03.315580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ecca94-cece-44c2-a2d3-8812be9ab675 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:03.315893 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ecca94-cece-44c2-a2d3-8812be9ab675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:03.316171 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ecca94-cece-44c2-a2d3-8812be9ab675 admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:03.316820 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ecca94-cece-44c2-a2d3-8812be9ab675 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:03.318804 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ecca94-cece-44c2-a2d3-8812be9ab675 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:03.354477 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1323/2648] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:41:03 2022] POST /identity/v3/projects => generated 367 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:41:03.356060 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ecca94-cece-44c2-a2d3-8812be9ab675 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:03.387396 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1326/2649] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Sep 1 16:41:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:41:03.395257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50508a21-6217-42b7-a603-20efbe90e00b None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:03.397659 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-50508a21-6217-42b7-a603-20efbe90e00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:03.429627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ecca94-cece-44c2-a2d3-8812be9ab675 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:03.435016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2849d-a175-4d34-96f2-a4620bfac36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:03.435665 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2849d-a175-4d34-96f2-a4620bfac36c admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:03.435903 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2849d-a175-4d34-96f2-a4620bfac36c admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:03.436176 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2849d-a175-4d34-96f2-a4620bfac36c admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:03.436975 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2849d-a175-4d34-96f2-a4620bfac36c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:03.438968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2849d-a175-4d34-96f2-a4620bfac36c admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:03.464863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1324/2650] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:41:03 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:03.468716 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2849d-a175-4d34-96f2-a4620bfac36c admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:03.479262 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2849d-a175-4d34-96f2-a4620bfac36c admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:03.489767 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50508a21-6217-42b7-a603-20efbe90e00b None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:03.496936 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73382c89-bd07-4679-aea5-1b2d6993c176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:03.497652 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73382c89-bd07-4679-aea5-1b2d6993c176 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:03.498000 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73382c89-bd07-4679-aea5-1b2d6993c176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:03.498358 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73382c89-bd07-4679-aea5-1b2d6993c176 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:03.499012 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73382c89-bd07-4679-aea5-1b2d6993c176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:03.500609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73382c89-bd07-4679-aea5-1b2d6993c176 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:03.514021 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1325/2651] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:03.517874 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73382c89-bd07-4679-aea5-1b2d6993c176 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:03.520135 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:03.530719 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73382c89-bd07-4679-aea5-1b2d6993c176 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:03.536003 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815307a-1ff0-45b4-9824-87ea2cd24d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:03.536657 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815307a-1ff0-45b4-9824-87ea2cd24d98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:03.536938 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815307a-1ff0-45b4-9824-87ea2cd24d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:03.537215 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815307a-1ff0-45b4-9824-87ea2cd24d98 admin admin] PATH_INFO: `/v3/projects/b4c0d6742908436bb76056aa16176417/users/5a18eb6c5e044c0885224daaedf40d21/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:03.556892 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1815307a-1ff0-45b4-9824-87ea2cd24d98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4c0d6742908436bb76056aa16176417, user_id=5a18eb6c5e044c0885224daaedf40d21, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:03.557747 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1815307a-1ff0-45b4-9824-87ea2cd24d98 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:03.567422 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d43b4-27bd-4c5c-873b-c2cbd32ba60f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8137f6b2ced040bd9752239c7e987f03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:03.568459 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d43b4-27bd-4c5c-873b-c2cbd32ba60f service swift] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:03.568894 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d43b4-27bd-4c5c-873b-c2cbd32ba60f service swift] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:03.569329 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d43b4-27bd-4c5c-873b-c2cbd32ba60f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:03.571594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1326/2652] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:41:03 2022] PUT /identity/v3/projects/b4c0d6742908436bb76056aa16176417/users/5a18eb6c5e044c0885224daaedf40d21/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:41:03.575641 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815307a-1ff0-45b4-9824-87ea2cd24d98 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:03.617227 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:03.647212 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d43b4-27bd-4c5c-873b-c2cbd32ba60f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:03.649784 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d43b4-27bd-4c5c-873b-c2cbd32ba60f service swift] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:03.656304 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815307a-1ff0-45b4-9824-87ea2cd24d98 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:03.663628 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1327/2653] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 1 16:41:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 16:41:03.672302 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4d311-f9b1-4759-8a00-07eb7d92e5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:03.673204 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4d311-f9b1-4759-8a00-07eb7d92e5fb admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:03.673563 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4d311-f9b1-4759-8a00-07eb7d92e5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:03.673918 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4d311-f9b1-4759-8a00-07eb7d92e5fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:03.674692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4d311-f9b1-4759-8a00-07eb7d92e5fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:03.677103 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4d311-f9b1-4759-8a00-07eb7d92e5fb admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:03.691229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1327/2654] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:41:03.695584 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d43b4-27bd-4c5c-873b-c2cbd32ba60f service swift] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:03.719049 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d43b4-27bd-4c5c-873b-c2cbd32ba60f service swift] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:03.727210 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eaceda-0d9a-4bc0-aca3-23ad826ba6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:03.727890 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eaceda-0d9a-4bc0-aca3-23ad826ba6db admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:03.728171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eaceda-0d9a-4bc0-aca3-23ad826ba6db admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:03.728616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eaceda-0d9a-4bc0-aca3-23ad826ba6db admin admin] PATH_INFO: `/v3/projects/b4c0d6742908436bb76056aa16176417/users/5a18eb6c5e044c0885224daaedf40d21/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:03.730475 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eaceda-0d9a-4bc0-aca3-23ad826ba6db admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4c0d6742908436bb76056aa16176417, user_id=5a18eb6c5e044c0885224daaedf40d21, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:03.733316 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eaceda-0d9a-4bc0-aca3-23ad826ba6db admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:03.749656 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1328/2655] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:41:03 2022] PUT /identity/v3/projects/b4c0d6742908436bb76056aa16176417/users/5a18eb6c5e044c0885224daaedf40d21/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:03.761178 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d83630-85b5-4b7a-bafd-7819b4ca57ab None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:03.761802 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d83630-85b5-4b7a-bafd-7819b4ca57ab None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:03.762351 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d83630-85b5-4b7a-bafd-7819b4ca57ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:03.844132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-c5d83630-85b5-4b7a-bafd-7819b4ca57ab None None] MFA Rules not processed for user `5a18eb6c5e044c0885224daaedf40d21`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:03.845383 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-c5d83630-85b5-4b7a-bafd-7819b4ca57ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:04.035579 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1328/2656] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:03 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:04.047547 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eaceda-0d9a-4bc0-aca3-23ad826ba6db admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:04.049531 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-84eaceda-0d9a-4bc0-aca3-23ad826ba6db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:04.154756 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eaceda-0d9a-4bc0-aca3-23ad826ba6db admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:04.159377 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acfb42f-1084-46d4-bec7-8bc54c942036 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:04.160280 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acfb42f-1084-46d4-bec7-8bc54c942036 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:04.160631 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acfb42f-1084-46d4-bec7-8bc54c942036 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:04.161038 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acfb42f-1084-46d4-bec7-8bc54c942036 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:04.174332 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acfb42f-1084-46d4-bec7-8bc54c942036 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:04.176000 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acfb42f-1084-46d4-bec7-8bc54c942036 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:04.188298 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1329/2657] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:04 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:41:04.254574 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d83630-85b5-4b7a-bafd-7819b4ca57ab None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:04.256783 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-c5d83630-85b5-4b7a-bafd-7819b4ca57ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:04.277535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d83630-85b5-4b7a-bafd-7819b4ca57ab None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:04.289188 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaec552-c762-4f6c-b65d-d580a5e3437d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:04.290093 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaec552-c762-4f6c-b65d-d580a5e3437d service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:04.290576 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaec552-c762-4f6c-b65d-d580a5e3437d service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:04.291075 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaec552-c762-4f6c-b65d-d580a5e3437d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:04.409266 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaec552-c762-4f6c-b65d-d580a5e3437d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:04.411092 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaec552-c762-4f6c-b65d-d580a5e3437d service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:04.422840 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1329/2658] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:07.977146 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acfb42f-1084-46d4-bec7-8bc54c942036 service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:08.039396 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:08.044517 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acfb42f-1084-46d4-bec7-8bc54c942036 service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:08.050607 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f898673-8139-401f-9dbd-8466f0f58e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:08.051420 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f898673-8139-401f-9dbd-8466f0f58e2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:08.051706 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f898673-8139-401f-9dbd-8466f0f58e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:08.051976 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f898673-8139-401f-9dbd-8466f0f58e2c admin admin] PATH_INFO: `/v3/users/087f321f6997400b847fe1263af9f89e` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:08.053168 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f898673-8139-401f-9dbd-8466f0f58e2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=087f321f6997400b847fe1263af9f89e)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:08.055139 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f898673-8139-401f-9dbd-8466f0f58e2c admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:08.113754 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1f898673-8139-401f-9dbd-8466f0f58e2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '087f321f6997400b847fe1263af9f89e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:08.123558 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1f898673-8139-401f-9dbd-8466f0f58e2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '087f321f6997400b847fe1263af9f89e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:08.124139 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1f898673-8139-401f-9dbd-8466f0f58e2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '087f321f6997400b847fe1263af9f89e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:08.138637 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1f898673-8139-401f-9dbd-8466f0f58e2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '087f321f6997400b847fe1263af9f89e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:08.139342 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1f898673-8139-401f-9dbd-8466f0f58e2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '087f321f6997400b847fe1263af9f89e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:08.151732 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1330/2659] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:41:07 2022] DELETE /identity/v3/users/087f321f6997400b847fe1263af9f89e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:08.156489 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0f719-8415-4a59-9118-65fe54df9bb5 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:08.157041 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0f719-8415-4a59-9118-65fe54df9bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:08.157041 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0f719-8415-4a59-9118-65fe54df9bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:08.177752 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-80a0f719-8415-4a59-9118-65fe54df9bb5 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:08.179014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-80a0f719-8415-4a59-9118-65fe54df9bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:08.231405 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=100 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:08.245953 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1330/2660] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:08.257914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f898673-8139-401f-9dbd-8466f0f58e2c admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:08.259748 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-1f898673-8139-401f-9dbd-8466f0f58e2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:08.330872 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f898673-8139-401f-9dbd-8466f0f58e2c admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:08.334616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b1d637-188d-44cf-adc1-a2ccafff3665 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:08.335231 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1d637-188d-44cf-adc1-a2ccafff3665 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:08.335508 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1d637-188d-44cf-adc1-a2ccafff3665 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:08.335776 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1d637-188d-44cf-adc1-a2ccafff3665 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:08.345439 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b1d637-188d-44cf-adc1-a2ccafff3665 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:08.348726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b1d637-188d-44cf-adc1-a2ccafff3665 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:08.358150 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1331/2661] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:08.642592 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0f719-8415-4a59-9118-65fe54df9bb5 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:08.644542 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-80a0f719-8415-4a59-9118-65fe54df9bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:08.657737 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0f719-8415-4a59-9118-65fe54df9bb5 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:08.663058 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8fc50-1ecb-4fb9-bd80-6dccb99d0ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:08.663820 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8fc50-1ecb-4fb9-bd80-6dccb99d0ca6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:08.664220 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8fc50-1ecb-4fb9-bd80-6dccb99d0ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:08.664601 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8fc50-1ecb-4fb9-bd80-6dccb99d0ca6 admin admin] PATH_INFO: `/v3/projects/63a01bf1c1294eb793db2c2cce4df3f4` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:08.665839 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b8fc50-1ecb-4fb9-bd80-6dccb99d0ca6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63a01bf1c1294eb793db2c2cce4df3f4)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:08.668489 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b8fc50-1ecb-4fb9-bd80-6dccb99d0ca6 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:08.728240 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-a9b8fc50-1ecb-4fb9-bd80-6dccb99d0ca6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63a01bf1c1294eb793db2c2cce4df3f4', 'request_id': 'req-a9b8fc50-1ecb-4fb9-bd80-6dccb99d0ca6'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:08.742738 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-a9b8fc50-1ecb-4fb9-bd80-6dccb99d0ca6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63a01bf1c1294eb793db2c2cce4df3f4', 'request_id': 'req-a9b8fc50-1ecb-4fb9-bd80-6dccb99d0ca6'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:08.750107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-a9b8fc50-1ecb-4fb9-bd80-6dccb99d0ca6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63a01bf1c1294eb793db2c2cce4df3f4', 'request_id': 'req-a9b8fc50-1ecb-4fb9-bd80-6dccb99d0ca6'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:08.764051 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-a9b8fc50-1ecb-4fb9-bd80-6dccb99d0ca6 admin admin] The token cache is being invalidate because project 63a01bf1c1294eb793db2c2cce4df3f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:41:08.764399 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-a9b8fc50-1ecb-4fb9-bd80-6dccb99d0ca6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:08.765256 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-a9b8fc50-1ecb-4fb9-bd80-6dccb99d0ca6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:08.767042 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1331/2662] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:41:08 2022] DELETE /identity/v3/projects/63a01bf1c1294eb793db2c2cce4df3f4 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:08.770875 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b1d637-188d-44cf-adc1-a2ccafff3665 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:08.781428 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b1d637-188d-44cf-adc1-a2ccafff3665 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:08.864309 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24229e3b-1d78-4fa9-9691-01fad9c5c7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:08.865154 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24229e3b-1d78-4fa9-9691-01fad9c5c7ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:08.865535 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24229e3b-1d78-4fa9-9691-01fad9c5c7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:08.865933 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24229e3b-1d78-4fa9-9691-01fad9c5c7ff admin admin] PATH_INFO: `/v3/users/5a18eb6c5e044c0885224daaedf40d21` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:08.867196 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24229e3b-1d78-4fa9-9691-01fad9c5c7ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a18eb6c5e044c0885224daaedf40d21)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:08.869759 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24229e3b-1d78-4fa9-9691-01fad9c5c7ff admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:08.897775 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:08.936470 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-24229e3b-1d78-4fa9-9691-01fad9c5c7ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a18eb6c5e044c0885224daaedf40d21'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:08.944973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-24229e3b-1d78-4fa9-9691-01fad9c5c7ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a18eb6c5e044c0885224daaedf40d21'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:08.945622 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-24229e3b-1d78-4fa9-9691-01fad9c5c7ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a18eb6c5e044c0885224daaedf40d21'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:08.958349 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-24229e3b-1d78-4fa9-9691-01fad9c5c7ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a18eb6c5e044c0885224daaedf40d21'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:08.959066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-24229e3b-1d78-4fa9-9691-01fad9c5c7ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a18eb6c5e044c0885224daaedf40d21'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:08.971416 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1332/2663] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:41:08 2022] DELETE /identity/v3/users/5a18eb6c5e044c0885224daaedf40d21 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:09.274973 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8fc50-1ecb-4fb9-bd80-6dccb99d0ca6 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:09.277004 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-a9b8fc50-1ecb-4fb9-bd80-6dccb99d0ca6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:09.291548 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8fc50-1ecb-4fb9-bd80-6dccb99d0ca6 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:09.368577 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e33c5-83ea-49d4-8eb9-67eed77f0b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:09.370253 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e33c5-83ea-49d4-8eb9-67eed77f0b3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:09.370551 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e33c5-83ea-49d4-8eb9-67eed77f0b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:09.370839 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e33c5-83ea-49d4-8eb9-67eed77f0b3b admin admin] PATH_INFO: `/v3/projects/b4c0d6742908436bb76056aa16176417` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:09.372028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789e33c5-83ea-49d4-8eb9-67eed77f0b3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4c0d6742908436bb76056aa16176417)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:09.374041 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789e33c5-83ea-49d4-8eb9-67eed77f0b3b admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:09.432180 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-789e33c5-83ea-49d4-8eb9-67eed77f0b3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4c0d6742908436bb76056aa16176417', 'request_id': 'req-789e33c5-83ea-49d4-8eb9-67eed77f0b3b'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:09.440514 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-789e33c5-83ea-49d4-8eb9-67eed77f0b3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4c0d6742908436bb76056aa16176417', 'request_id': 'req-789e33c5-83ea-49d4-8eb9-67eed77f0b3b'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:09.447497 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-789e33c5-83ea-49d4-8eb9-67eed77f0b3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4c0d6742908436bb76056aa16176417', 'request_id': 'req-789e33c5-83ea-49d4-8eb9-67eed77f0b3b'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:09.459692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-789e33c5-83ea-49d4-8eb9-67eed77f0b3b admin admin] The token cache is being invalidate because project b4c0d6742908436bb76056aa16176417 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:41:09.459995 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-789e33c5-83ea-49d4-8eb9-67eed77f0b3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:09.460657 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-789e33c5-83ea-49d4-8eb9-67eed77f0b3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:09.462142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1332/2664] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:41:09 2022] DELETE /identity/v3/projects/b4c0d6742908436bb76056aa16176417 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:09.467537 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38d57a-e6bf-4cc9-aea6-7e408d5c0add None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:09.468062 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38d57a-e6bf-4cc9-aea6-7e408d5c0add None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:09.468212 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38d57a-e6bf-4cc9-aea6-7e408d5c0add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:09.491461 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-9c38d57a-e6bf-4cc9-aea6-7e408d5c0add None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:09.492717 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9c38d57a-e6bf-4cc9-aea6-7e408d5c0add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:09.555531 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1333/2665] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:09.563255 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c40af-8ff5-4e9b-889a-edfd1454a3c4 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:09.563563 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c40af-8ff5-4e9b-889a-edfd1454a3c4 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:09.563838 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c40af-8ff5-4e9b-889a-edfd1454a3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:09.579896 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:09.587234 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-968c40af-8ff5-4e9b-889a-edfd1454a3c4 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:09.590135 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-968c40af-8ff5-4e9b-889a-edfd1454a3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:09.602194 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1333/2666] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:09.603994 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38d57a-e6bf-4cc9-aea6-7e408d5c0add None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:09.614067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38d57a-e6bf-4cc9-aea6-7e408d5c0add None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:09.621182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c307c-e6c5-4ff6-8096-61653198fb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:09.621808 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c307c-e6c5-4ff6-8096-61653198fb80 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:09.622088 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c307c-e6c5-4ff6-8096-61653198fb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:09.622404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c307c-e6c5-4ff6-8096-61653198fb80 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:09.623063 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c307c-e6c5-4ff6-8096-61653198fb80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:41:09.623368 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c307c-e6c5-4ff6-8096-61653198fb80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:09.625017 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c307c-e6c5-4ff6-8096-61653198fb80 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:09.638793 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1334/2667] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:41:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:41:09.643399 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c40af-8ff5-4e9b-889a-edfd1454a3c4 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:09.654484 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c40af-8ff5-4e9b-889a-edfd1454a3c4 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:09.659232 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5fbfa-5218-47ce-8450-1561a6062bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:09.659866 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5fbfa-5218-47ce-8450-1561a6062bd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:09.660153 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5fbfa-5218-47ce-8450-1561a6062bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:09.660433 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5fbfa-5218-47ce-8450-1561a6062bd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:09.661216 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e5fbfa-5218-47ce-8450-1561a6062bd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:09.663612 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e5fbfa-5218-47ce-8450-1561a6062bd2 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:09.693682 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1334/2668] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:41:09 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:09.698578 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c307c-e6c5-4ff6-8096-61653198fb80 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:09.708768 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c307c-e6c5-4ff6-8096-61653198fb80 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:09.769697 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd13e90-694b-492d-af14-a2241b1172bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:09.770575 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd13e90-694b-492d-af14-a2241b1172bd admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:09.771021 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd13e90-694b-492d-af14-a2241b1172bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:09.771390 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd13e90-694b-492d-af14-a2241b1172bd admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:09.772179 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd13e90-694b-492d-af14-a2241b1172bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:09.774399 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd13e90-694b-492d-af14-a2241b1172bd admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:09.797704 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1335/2669] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:41:09 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:41:09.802044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5fbfa-5218-47ce-8450-1561a6062bd2 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:09.814126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5fbfa-5218-47ce-8450-1561a6062bd2 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:09.818981 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3088fb-2d88-4d5f-9a58-c8c81c828dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:09.819704 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3088fb-2d88-4d5f-9a58-c8c81c828dc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:09.820047 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3088fb-2d88-4d5f-9a58-c8c81c828dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:09.820386 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3088fb-2d88-4d5f-9a58-c8c81c828dc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:09.821098 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3088fb-2d88-4d5f-9a58-c8c81c828dc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:09.823353 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3088fb-2d88-4d5f-9a58-c8c81c828dc0 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:09.834304 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1335/2670] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:41:09.839181 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd13e90-694b-492d-af14-a2241b1172bd admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:09.849505 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd13e90-694b-492d-af14-a2241b1172bd admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:09.863945 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840c305-947c-4377-b1ad-7b71817cf773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:09.864846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840c305-947c-4377-b1ad-7b71817cf773 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:09.865071 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840c305-947c-4377-b1ad-7b71817cf773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:09.865305 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840c305-947c-4377-b1ad-7b71817cf773 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:09.865999 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c840c305-947c-4377-b1ad-7b71817cf773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:09.867621 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c840c305-947c-4377-b1ad-7b71817cf773 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:09.885956 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1336/2671] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:41:09.890616 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3088fb-2d88-4d5f-9a58-c8c81c828dc0 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:09.903457 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3088fb-2d88-4d5f-9a58-c8c81c828dc0 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:09.908216 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6319a-89da-4df3-b21b-09a53668b7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:09.908957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6319a-89da-4df3-b21b-09a53668b7a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:09.909307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6319a-89da-4df3-b21b-09a53668b7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:09.909659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6319a-89da-4df3-b21b-09a53668b7a3 admin admin] PATH_INFO: `/v3/projects/8665c42b8e9d4de49dce20280be9d9a1/users/c74770032b314d5486863c1f939f6e60/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:09.934952 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e6319a-89da-4df3-b21b-09a53668b7a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8665c42b8e9d4de49dce20280be9d9a1, user_id=c74770032b314d5486863c1f939f6e60, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:09.937800 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e6319a-89da-4df3-b21b-09a53668b7a3 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:09.964965 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1336/2672] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:41:09 2022] PUT /identity/v3/projects/8665c42b8e9d4de49dce20280be9d9a1/users/c74770032b314d5486863c1f939f6e60/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:09.966680 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b310d16-0acb-4df1-a090-615d98050c17 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:09.966969 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b310d16-0acb-4df1-a090-615d98050c17 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:09.967254 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b310d16-0acb-4df1-a090-615d98050c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:09.987823 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:10.017629 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-3b310d16-0acb-4df1-a090-615d98050c17 None None] MFA Rules not processed for user `c74770032b314d5486863c1f939f6e60`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:10.018888 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-3b310d16-0acb-4df1-a090-615d98050c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:10.110484 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1337/2673] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:10.114506 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6319a-89da-4df3-b21b-09a53668b7a3 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:10.199328 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6319a-89da-4df3-b21b-09a53668b7a3 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:10.203867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4450cef-4eaa-4cd4-9573-b50d3bf38e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:10.204543 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4450cef-4eaa-4cd4-9573-b50d3bf38e94 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:10.204828 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4450cef-4eaa-4cd4-9573-b50d3bf38e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:10.205101 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4450cef-4eaa-4cd4-9573-b50d3bf38e94 admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:10.205736 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4450cef-4eaa-4cd4-9573-b50d3bf38e94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:10.207882 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4450cef-4eaa-4cd4-9573-b50d3bf38e94 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:10.235185 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1337/2674] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:41:10 2022] POST /identity/v3/projects => generated 381 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:10.239380 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b310d16-0acb-4df1-a090-615d98050c17 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:10.251261 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b310d16-0acb-4df1-a090-615d98050c17 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:10.319397 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c1212-18cd-4960-abf5-7f6d47ac8bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:10.320030 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c1212-18cd-4960-abf5-7f6d47ac8bda admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:10.320311 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c1212-18cd-4960-abf5-7f6d47ac8bda admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:10.320719 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c1212-18cd-4960-abf5-7f6d47ac8bda admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:10.321404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c1212-18cd-4960-abf5-7f6d47ac8bda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:10.323434 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c1212-18cd-4960-abf5-7f6d47ac8bda admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:10.351327 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1338/2675] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:41:10 2022] POST /identity/v3/users => generated 297 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:10.354633 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4450cef-4eaa-4cd4-9573-b50d3bf38e94 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:10.375292 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4450cef-4eaa-4cd4-9573-b50d3bf38e94 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:10.381156 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250fd119-7ab2-48f1-8e5a-8d26029f55b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:10.381860 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fd119-7ab2-48f1-8e5a-8d26029f55b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:10.382144 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fd119-7ab2-48f1-8e5a-8d26029f55b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:10.382448 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fd119-7ab2-48f1-8e5a-8d26029f55b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:10.383100 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250fd119-7ab2-48f1-8e5a-8d26029f55b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:10.385612 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250fd119-7ab2-48f1-8e5a-8d26029f55b4 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:10.398749 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1338/2676] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:10.403355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c1212-18cd-4960-abf5-7f6d47ac8bda admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:10.413579 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c1212-18cd-4960-abf5-7f6d47ac8bda admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:10.418581 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140afb4-db2a-4af2-9855-a8b989f58159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:10.419238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140afb4-db2a-4af2-9855-a8b989f58159 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:10.419532 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140afb4-db2a-4af2-9855-a8b989f58159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:10.419811 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140afb4-db2a-4af2-9855-a8b989f58159 admin admin] PATH_INFO: `/v3/projects/ac05878c573943e28a30d2ae269fc5c1/users/6a6ec2dda7564d1986869546dd4eb72d/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:10.435790 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140afb4-db2a-4af2-9855-a8b989f58159 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac05878c573943e28a30d2ae269fc5c1, user_id=6a6ec2dda7564d1986869546dd4eb72d, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:10.438497 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140afb4-db2a-4af2-9855-a8b989f58159 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:10.453555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1339/2677] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:41:10 2022] PUT /identity/v3/projects/ac05878c573943e28a30d2ae269fc5c1/users/6a6ec2dda7564d1986869546dd4eb72d/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:41:10.457583 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250fd119-7ab2-48f1-8e5a-8d26029f55b4 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:10.468675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250fd119-7ab2-48f1-8e5a-8d26029f55b4 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:10.530091 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486426aa-5611-404e-87bc-888eb3591f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:10.530743 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486426aa-5611-404e-87bc-888eb3591f40 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:10.531023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486426aa-5611-404e-87bc-888eb3591f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:10.531292 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486426aa-5611-404e-87bc-888eb3591f40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:10.531964 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486426aa-5611-404e-87bc-888eb3591f40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:10.533617 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486426aa-5611-404e-87bc-888eb3591f40 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:10.544314 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1339/2678] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:10.549042 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140afb4-db2a-4af2-9855-a8b989f58159 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:10.559437 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140afb4-db2a-4af2-9855-a8b989f58159 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:10.564083 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf428d04-b138-488c-9a69-86298b58bfaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:10.564809 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf428d04-b138-488c-9a69-86298b58bfaf admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:10.565157 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf428d04-b138-488c-9a69-86298b58bfaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:10.565496 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf428d04-b138-488c-9a69-86298b58bfaf admin admin] PATH_INFO: `/v3/projects/ac05878c573943e28a30d2ae269fc5c1/users/6a6ec2dda7564d1986869546dd4eb72d/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:10.567560 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf428d04-b138-488c-9a69-86298b58bfaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac05878c573943e28a30d2ae269fc5c1, user_id=6a6ec2dda7564d1986869546dd4eb72d, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:10.570302 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf428d04-b138-488c-9a69-86298b58bfaf admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:10.581794 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1340/2679] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:41:10 2022] PUT /identity/v3/projects/ac05878c573943e28a30d2ae269fc5c1/users/6a6ec2dda7564d1986869546dd4eb72d/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:41:10.592825 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fa8ec-e217-43c5-8925-251fae3cf7ca None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:10.593179 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fa8ec-e217-43c5-8925-251fae3cf7ca None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:10.593534 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fa8ec-e217-43c5-8925-251fae3cf7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:10.622119 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:10.625827 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-7c1fa8ec-e217-43c5-8925-251fae3cf7ca None None] MFA Rules not processed for user `6a6ec2dda7564d1986869546dd4eb72d`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:10.627649 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-7c1fa8ec-e217-43c5-8925-251fae3cf7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:10.710914 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1340/2680] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:10 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:10.722185 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf428d04-b138-488c-9a69-86298b58bfaf admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:10.723999 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-cf428d04-b138-488c-9a69-86298b58bfaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:10.780800 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:10.786571 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf428d04-b138-488c-9a69-86298b58bfaf admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:10.791971 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e7c54-e378-458c-b77a-5c88863c1b33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:10.792984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e7c54-e378-458c-b77a-5c88863c1b33 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:10.793445 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e7c54-e378-458c-b77a-5c88863c1b33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:10.793820 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e7c54-e378-458c-b77a-5c88863c1b33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:10.804371 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e7c54-e378-458c-b77a-5c88863c1b33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:10.805946 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e7c54-e378-458c-b77a-5c88863c1b33 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:10.816667 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1341/2681] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:10 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:10.886756 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1fa8ec-e217-43c5-8925-251fae3cf7ca None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:10.972861 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1fa8ec-e217-43c5-8925-251fae3cf7ca None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:10.978973 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca87c75-0b5e-4dc3-ae43-e8f3c87a68ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:10.979616 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca87c75-0b5e-4dc3-ae43-e8f3c87a68ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:10.979897 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca87c75-0b5e-4dc3-ae43-e8f3c87a68ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:10.980169 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca87c75-0b5e-4dc3-ae43-e8f3c87a68ee admin admin] PATH_INFO: `/v3/users/c74770032b314d5486863c1f939f6e60` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:10.981336 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca87c75-0b5e-4dc3-ae43-e8f3c87a68ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=c74770032b314d5486863c1f939f6e60)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:10.983317 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca87c75-0b5e-4dc3-ae43-e8f3c87a68ee admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:11.040646 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-eca87c75-0b5e-4dc3-ae43-e8f3c87a68ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c74770032b314d5486863c1f939f6e60'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:11.041941 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-eca87c75-0b5e-4dc3-ae43-e8f3c87a68ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c74770032b314d5486863c1f939f6e60'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:11.051488 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-eca87c75-0b5e-4dc3-ae43-e8f3c87a68ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c74770032b314d5486863c1f939f6e60'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:11.052185 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-eca87c75-0b5e-4dc3-ae43-e8f3c87a68ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c74770032b314d5486863c1f939f6e60'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:11.067098 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-eca87c75-0b5e-4dc3-ae43-e8f3c87a68ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c74770032b314d5486863c1f939f6e60'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:11.081092 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1341/2682] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:41:10 2022] DELETE /identity/v3/users/c74770032b314d5486863c1f939f6e60 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:11.085958 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e305c5-cbce-4614-a338-eba816672353 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:11.086262 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e305c5-cbce-4614-a338-eba816672353 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:11.086554 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e305c5-cbce-4614-a338-eba816672353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:11.107898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-d5e305c5-cbce-4614-a338-eba816672353 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:11.109158 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d5e305c5-cbce-4614-a338-eba816672353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:11.185558 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1342/2683] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:11.197807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca87c75-0b5e-4dc3-ae43-e8f3c87a68ee admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:11.199449 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-eca87c75-0b5e-4dc3-ae43-e8f3c87a68ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:11.227925 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:11.289438 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca87c75-0b5e-4dc3-ae43-e8f3c87a68ee admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:11.293262 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e69e5-eaa2-43eb-8b54-922d81bc3ef7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:11.293887 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e69e5-eaa2-43eb-8b54-922d81bc3ef7 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:11.294173 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e69e5-eaa2-43eb-8b54-922d81bc3ef7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:11.294497 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e69e5-eaa2-43eb-8b54-922d81bc3ef7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:11.304292 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541e69e5-eaa2-43eb-8b54-922d81bc3ef7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:11.305746 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541e69e5-eaa2-43eb-8b54-922d81bc3ef7 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:11.317803 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1342/2684] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:11.655436 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e305c5-cbce-4614-a338-eba816672353 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:11.658349 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d5e305c5-cbce-4614-a338-eba816672353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:11.672082 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e305c5-cbce-4614-a338-eba816672353 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:11.677003 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea7f77f-3d20-4647-9754-ebc18146c1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:11.677735 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7f77f-3d20-4647-9754-ebc18146c1d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:11.677905 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7f77f-3d20-4647-9754-ebc18146c1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:11.678176 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7f77f-3d20-4647-9754-ebc18146c1d1 admin admin] PATH_INFO: `/v3/projects/8665c42b8e9d4de49dce20280be9d9a1` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:11.679324 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea7f77f-3d20-4647-9754-ebc18146c1d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8665c42b8e9d4de49dce20280be9d9a1)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:11.681435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea7f77f-3d20-4647-9754-ebc18146c1d1 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:11.743488 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0ea7f77f-3d20-4647-9754-ebc18146c1d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8665c42b8e9d4de49dce20280be9d9a1', 'request_id': 'req-0ea7f77f-3d20-4647-9754-ebc18146c1d1'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:11.751065 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0ea7f77f-3d20-4647-9754-ebc18146c1d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8665c42b8e9d4de49dce20280be9d9a1', 'request_id': 'req-0ea7f77f-3d20-4647-9754-ebc18146c1d1'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:11.760972 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0ea7f77f-3d20-4647-9754-ebc18146c1d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8665c42b8e9d4de49dce20280be9d9a1', 'request_id': 'req-0ea7f77f-3d20-4647-9754-ebc18146c1d1'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:11.774892 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0ea7f77f-3d20-4647-9754-ebc18146c1d1 admin admin] The token cache is being invalidate because project 8665c42b8e9d4de49dce20280be9d9a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:41:11.775799 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0ea7f77f-3d20-4647-9754-ebc18146c1d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:11.776512 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0ea7f77f-3d20-4647-9754-ebc18146c1d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:11.778314 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1343/2685] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:41:11 2022] DELETE /identity/v3/projects/8665c42b8e9d4de49dce20280be9d9a1 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:11.782048 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e69e5-eaa2-43eb-8b54-922d81bc3ef7 service neutron] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:11.792798 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e69e5-eaa2-43eb-8b54-922d81bc3ef7 service neutron] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:11.850313 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60795c8d-3e21-462f-bed7-b83b26379f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:11.850953 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60795c8d-3e21-462f-bed7-b83b26379f06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:11.851231 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60795c8d-3e21-462f-bed7-b83b26379f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:11.851500 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60795c8d-3e21-462f-bed7-b83b26379f06 admin admin] PATH_INFO: `/v3/users/6a6ec2dda7564d1986869546dd4eb72d` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:11.853141 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60795c8d-3e21-462f-bed7-b83b26379f06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a6ec2dda7564d1986869546dd4eb72d)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:11.856262 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60795c8d-3e21-462f-bed7-b83b26379f06 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:11.913124 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-60795c8d-3e21-462f-bed7-b83b26379f06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a6ec2dda7564d1986869546dd4eb72d'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:11.913803 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-60795c8d-3e21-462f-bed7-b83b26379f06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a6ec2dda7564d1986869546dd4eb72d'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:11.923906 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-60795c8d-3e21-462f-bed7-b83b26379f06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a6ec2dda7564d1986869546dd4eb72d'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:11.924848 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-60795c8d-3e21-462f-bed7-b83b26379f06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a6ec2dda7564d1986869546dd4eb72d'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:11.941094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-60795c8d-3e21-462f-bed7-b83b26379f06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a6ec2dda7564d1986869546dd4eb72d'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:11.958658 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1343/2686] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:41:11 2022] DELETE /identity/v3/users/6a6ec2dda7564d1986869546dd4eb72d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:41:12.279208 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea7f77f-3d20-4647-9754-ebc18146c1d1 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:12.281720 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0ea7f77f-3d20-4647-9754-ebc18146c1d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:12.294055 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea7f77f-3d20-4647-9754-ebc18146c1d1 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:12.381047 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922bf00c-4909-42b0-9f84-cf55117f9e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:12.381814 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bf00c-4909-42b0-9f84-cf55117f9e96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:12.382020 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bf00c-4909-42b0-9f84-cf55117f9e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:12.382286 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bf00c-4909-42b0-9f84-cf55117f9e96 admin admin] PATH_INFO: `/v3/projects/ac05878c573943e28a30d2ae269fc5c1` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:12.383631 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922bf00c-4909-42b0-9f84-cf55117f9e96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac05878c573943e28a30d2ae269fc5c1)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:12.386187 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922bf00c-4909-42b0-9f84-cf55117f9e96 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:12.403548 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:12.436653 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-922bf00c-4909-42b0-9f84-cf55117f9e96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac05878c573943e28a30d2ae269fc5c1', 'request_id': 'req-922bf00c-4909-42b0-9f84-cf55117f9e96'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:12.443090 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-922bf00c-4909-42b0-9f84-cf55117f9e96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac05878c573943e28a30d2ae269fc5c1', 'request_id': 'req-922bf00c-4909-42b0-9f84-cf55117f9e96'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:12.451689 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-922bf00c-4909-42b0-9f84-cf55117f9e96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac05878c573943e28a30d2ae269fc5c1', 'request_id': 'req-922bf00c-4909-42b0-9f84-cf55117f9e96'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:12.467423 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-922bf00c-4909-42b0-9f84-cf55117f9e96 admin admin] The token cache is being invalidate because project ac05878c573943e28a30d2ae269fc5c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:41:12.467719 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-922bf00c-4909-42b0-9f84-cf55117f9e96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:12.468961 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-922bf00c-4909-42b0-9f84-cf55117f9e96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:12.470566 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1344/2687] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:41:12 2022] DELETE /identity/v3/projects/ac05878c573943e28a30d2ae269fc5c1 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:12.475993 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90303d-5221-452e-bd9d-7e813005e682 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:12.476937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90303d-5221-452e-bd9d-7e813005e682 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:12.477519 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90303d-5221-452e-bd9d-7e813005e682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:12.498883 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-da90303d-5221-452e-bd9d-7e813005e682 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:12.500462 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-da90303d-5221-452e-bd9d-7e813005e682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:12.541739 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:12.573213 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1344/2688] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:12.581941 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512801a-97e0-4855-902c-6568e296b123 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:12.582226 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512801a-97e0-4855-902c-6568e296b123 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:12.582534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512801a-97e0-4855-902c-6568e296b123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:12.602179 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-e512801a-97e0-4855-902c-6568e296b123 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:12.603451 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-e512801a-97e0-4855-902c-6568e296b123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:12.611228 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1345/2689] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:12.615303 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90303d-5221-452e-bd9d-7e813005e682 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:12.625533 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90303d-5221-452e-bd9d-7e813005e682 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:12.630121 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5924b44-22eb-4e03-a6cd-c4952fc2df30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:12.630755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5924b44-22eb-4e03-a6cd-c4952fc2df30 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:12.631042 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5924b44-22eb-4e03-a6cd-c4952fc2df30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:12.631319 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5924b44-22eb-4e03-a6cd-c4952fc2df30 admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:12.631962 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5924b44-22eb-4e03-a6cd-c4952fc2df30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:41:12.632266 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5924b44-22eb-4e03-a6cd-c4952fc2df30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:12.633860 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5924b44-22eb-4e03-a6cd-c4952fc2df30 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:12.645980 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1345/2690] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:41:12.650186 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512801a-97e0-4855-902c-6568e296b123 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:12.659163 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512801a-97e0-4855-902c-6568e296b123 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:12.663760 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd01c8-8ec4-4170-bf13-dc3ba981db52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:12.664340 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd01c8-8ec4-4170-bf13-dc3ba981db52 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:12.664616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd01c8-8ec4-4170-bf13-dc3ba981db52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:12.664888 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd01c8-8ec4-4170-bf13-dc3ba981db52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:12.665514 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd01c8-8ec4-4170-bf13-dc3ba981db52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:12.667974 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd01c8-8ec4-4170-bf13-dc3ba981db52 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:12.695204 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1346/2691] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:41:12 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:12.699861 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5924b44-22eb-4e03-a6cd-c4952fc2df30 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:12.709867 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5924b44-22eb-4e03-a6cd-c4952fc2df30 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:12.767197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c6c3d-e9d7-4ea3-9138-7b14156fa1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:12.767987 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c6c3d-e9d7-4ea3-9138-7b14156fa1ba admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:12.768123 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c6c3d-e9d7-4ea3-9138-7b14156fa1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:12.768327 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c6c3d-e9d7-4ea3-9138-7b14156fa1ba admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:12.768953 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c6c3d-e9d7-4ea3-9138-7b14156fa1ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:12.770895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c6c3d-e9d7-4ea3-9138-7b14156fa1ba admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:12.795966 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1346/2692] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:41:12 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:12.797677 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd01c8-8ec4-4170-bf13-dc3ba981db52 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:12.808135 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd01c8-8ec4-4170-bf13-dc3ba981db52 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:12.812699 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72826d9-b7cf-44c6-8406-2923b5488eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:12.813293 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72826d9-b7cf-44c6-8406-2923b5488eb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:12.813573 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72826d9-b7cf-44c6-8406-2923b5488eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:12.813905 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72826d9-b7cf-44c6-8406-2923b5488eb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:12.814579 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72826d9-b7cf-44c6-8406-2923b5488eb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:12.816144 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72826d9-b7cf-44c6-8406-2923b5488eb4 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:12.826373 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1347/2693] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:12.830267 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c6c3d-e9d7-4ea3-9138-7b14156fa1ba admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:12.841211 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c6c3d-e9d7-4ea3-9138-7b14156fa1ba admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:12.845809 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8b3cf-c4c5-4088-8603-0af109ea597c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:12.846438 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8b3cf-c4c5-4088-8603-0af109ea597c admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:12.846720 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8b3cf-c4c5-4088-8603-0af109ea597c admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:12.846995 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8b3cf-c4c5-4088-8603-0af109ea597c admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:12.847637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f8b3cf-c4c5-4088-8603-0af109ea597c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:12.849298 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f8b3cf-c4c5-4088-8603-0af109ea597c admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:12.859536 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1347/2694] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:12.863623 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72826d9-b7cf-44c6-8406-2923b5488eb4 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:12.873802 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72826d9-b7cf-44c6-8406-2923b5488eb4 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:12.878407 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b8e919-f9a9-47ae-a08b-22f90b031258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:12.879032 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8e919-f9a9-47ae-a08b-22f90b031258 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:12.879313 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8e919-f9a9-47ae-a08b-22f90b031258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:12.879588 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8e919-f9a9-47ae-a08b-22f90b031258 admin admin] PATH_INFO: `/v3/projects/2fe8a2a279f044af808d7814e29d0531/users/9b2df42a2b004b3796df6fbaf99b38e8/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:12.894257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b8e919-f9a9-47ae-a08b-22f90b031258 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fe8a2a279f044af808d7814e29d0531, user_id=9b2df42a2b004b3796df6fbaf99b38e8, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:12.896885 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b8e919-f9a9-47ae-a08b-22f90b031258 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:12.910053 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1348/2695] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:41:12 2022] PUT /identity/v3/projects/2fe8a2a279f044af808d7814e29d0531/users/9b2df42a2b004b3796df6fbaf99b38e8/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:41:12.920306 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641486e2-0398-4f93-86b6-a8330d0c9e37 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:12.920596 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641486e2-0398-4f93-86b6-a8330d0c9e37 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:12.920875 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641486e2-0398-4f93-86b6-a8330d0c9e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:12.948582 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-641486e2-0398-4f93-86b6-a8330d0c9e37 None None] MFA Rules not processed for user `9b2df42a2b004b3796df6fbaf99b38e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:12.949791 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-641486e2-0398-4f93-86b6-a8330d0c9e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:13.019623 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1348/2696] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:13.024076 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b8e919-f9a9-47ae-a08b-22f90b031258 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:13.088606 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b8e919-f9a9-47ae-a08b-22f90b031258 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:13.093103 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74af5d3-a7ce-4416-8248-30949c8b230c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:13.093746 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74af5d3-a7ce-4416-8248-30949c8b230c admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:13.094030 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74af5d3-a7ce-4416-8248-30949c8b230c admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:13.094336 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74af5d3-a7ce-4416-8248-30949c8b230c admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:13.094968 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74af5d3-a7ce-4416-8248-30949c8b230c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:13.096858 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74af5d3-a7ce-4416-8248-30949c8b230c admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:13.124067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1349/2697] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:41:13 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:41:13.128411 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641486e2-0398-4f93-86b6-a8330d0c9e37 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:13.139350 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641486e2-0398-4f93-86b6-a8330d0c9e37 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:13.173754 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:13.200702 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d084fd0-93a2-4338-b257-7da865350a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:13.201319 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d084fd0-93a2-4338-b257-7da865350a37 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:13.201606 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d084fd0-93a2-4338-b257-7da865350a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:13.201878 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d084fd0-93a2-4338-b257-7da865350a37 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:13.202552 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d084fd0-93a2-4338-b257-7da865350a37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:13.204483 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d084fd0-93a2-4338-b257-7da865350a37 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:13.229591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1349/2698] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:41:13 2022] POST /identity/v3/users => generated 300 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:41:13.233585 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74af5d3-a7ce-4416-8248-30949c8b230c admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:13.244550 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74af5d3-a7ce-4416-8248-30949c8b230c admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:13.249232 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d378f761-ddbd-487a-a6dc-f301ee87ce3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:13.249854 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d378f761-ddbd-487a-a6dc-f301ee87ce3c admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:13.250127 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d378f761-ddbd-487a-a6dc-f301ee87ce3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:13.250435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d378f761-ddbd-487a-a6dc-f301ee87ce3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:13.251079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d378f761-ddbd-487a-a6dc-f301ee87ce3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:13.252660 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d378f761-ddbd-487a-a6dc-f301ee87ce3c admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:13.265352 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1350/2699] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:13.270269 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d084fd0-93a2-4338-b257-7da865350a37 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:13.281771 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d084fd0-93a2-4338-b257-7da865350a37 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:13.286746 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ef170-08b4-4fd2-8901-e35f475aa0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:13.287483 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ef170-08b4-4fd2-8901-e35f475aa0ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:13.287765 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ef170-08b4-4fd2-8901-e35f475aa0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:13.288039 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ef170-08b4-4fd2-8901-e35f475aa0ce admin admin] PATH_INFO: `/v3/projects/1d0c8492cd9441a892bf0ddd5fa69d62/users/db7a9ffefc5047c7907ac941ceaadcbc/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:13.303580 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5ef170-08b4-4fd2-8901-e35f475aa0ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d0c8492cd9441a892bf0ddd5fa69d62, user_id=db7a9ffefc5047c7907ac941ceaadcbc, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:13.306309 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5ef170-08b4-4fd2-8901-e35f475aa0ce admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:13.320194 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1350/2700] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:41:13 2022] PUT /identity/v3/projects/1d0c8492cd9441a892bf0ddd5fa69d62/users/db7a9ffefc5047c7907ac941ceaadcbc/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:13.324172 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d378f761-ddbd-487a-a6dc-f301ee87ce3c admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:13.334480 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d378f761-ddbd-487a-a6dc-f301ee87ce3c admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:13.344052 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:13.401339 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1750af-ae0c-4135-9998-6eef046afdd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:13.401974 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1750af-ae0c-4135-9998-6eef046afdd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:13.402274 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1750af-ae0c-4135-9998-6eef046afdd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:13.402547 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1750af-ae0c-4135-9998-6eef046afdd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:13.403248 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1750af-ae0c-4135-9998-6eef046afdd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:13.404814 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1750af-ae0c-4135-9998-6eef046afdd4 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:13.414680 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1351/2701] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:13.418893 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ef170-08b4-4fd2-8901-e35f475aa0ce admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:13.439318 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ef170-08b4-4fd2-8901-e35f475aa0ce admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:13.444197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f58514-a839-480c-8277-96ca1b03dcf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:13.444957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f58514-a839-480c-8277-96ca1b03dcf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:13.445297 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f58514-a839-480c-8277-96ca1b03dcf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:13.445577 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f58514-a839-480c-8277-96ca1b03dcf4 admin admin] PATH_INFO: `/v3/projects/1d0c8492cd9441a892bf0ddd5fa69d62/users/db7a9ffefc5047c7907ac941ceaadcbc/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:13.448033 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f58514-a839-480c-8277-96ca1b03dcf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d0c8492cd9441a892bf0ddd5fa69d62, user_id=db7a9ffefc5047c7907ac941ceaadcbc, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:13.451033 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f58514-a839-480c-8277-96ca1b03dcf4 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:13.461786 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1351/2702] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:41:13 2022] PUT /identity/v3/projects/1d0c8492cd9441a892bf0ddd5fa69d62/users/db7a9ffefc5047c7907ac941ceaadcbc/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:13.474441 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abb5e3-4752-4ec6-a115-19971536f2a9 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:13.474949 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abb5e3-4752-4ec6-a115-19971536f2a9 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:13.475553 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abb5e3-4752-4ec6-a115-19971536f2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:13.521251 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-62abb5e3-4752-4ec6-a115-19971536f2a9 None None] MFA Rules not processed for user `db7a9ffefc5047c7907ac941ceaadcbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:13.522524 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-62abb5e3-4752-4ec6-a115-19971536f2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:13.602320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1352/2703] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:13 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:13.613235 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f58514-a839-480c-8277-96ca1b03dcf4 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:13.614971 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-59f58514-a839-480c-8277-96ca1b03dcf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:13.675475 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f58514-a839-480c-8277-96ca1b03dcf4 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:13.679281 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5cca7-f984-4df5-864d-6526baf8a831 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:13.679902 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5cca7-f984-4df5-864d-6526baf8a831 service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:13.680324 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5cca7-f984-4df5-864d-6526baf8a831 service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:13.680637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5cca7-f984-4df5-864d-6526baf8a831 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:13.691237 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5cca7-f984-4df5-864d-6526baf8a831 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:13.692717 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5cca7-f984-4df5-864d-6526baf8a831 service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:13.703044 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1352/2704] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:13 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:14.284164 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abb5e3-4752-4ec6-a115-19971536f2a9 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:14.351980 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abb5e3-4752-4ec6-a115-19971536f2a9 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:14.356144 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68634d89-69bb-4af5-8aef-eaef6077f0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:14.356854 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68634d89-69bb-4af5-8aef-eaef6077f0f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:14.357144 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68634d89-69bb-4af5-8aef-eaef6077f0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:14.357951 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68634d89-69bb-4af5-8aef-eaef6077f0f8 admin admin] PATH_INFO: `/v3/users/9b2df42a2b004b3796df6fbaf99b38e8` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:14.358907 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68634d89-69bb-4af5-8aef-eaef6077f0f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b2df42a2b004b3796df6fbaf99b38e8)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:14.360864 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68634d89-69bb-4af5-8aef-eaef6077f0f8 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:14.402216 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:14.412177 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-68634d89-69bb-4af5-8aef-eaef6077f0f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b2df42a2b004b3796df6fbaf99b38e8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:14.420689 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-68634d89-69bb-4af5-8aef-eaef6077f0f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b2df42a2b004b3796df6fbaf99b38e8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:14.421414 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-68634d89-69bb-4af5-8aef-eaef6077f0f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b2df42a2b004b3796df6fbaf99b38e8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:14.434364 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-68634d89-69bb-4af5-8aef-eaef6077f0f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b2df42a2b004b3796df6fbaf99b38e8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:14.435184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-68634d89-69bb-4af5-8aef-eaef6077f0f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b2df42a2b004b3796df6fbaf99b38e8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:14.450178 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1353/2705] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:41:14 2022] DELETE /identity/v3/users/9b2df42a2b004b3796df6fbaf99b38e8 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:14.454593 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cc84f-346e-4399-bb83-6159c027e6e3 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:14.455091 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cc84f-346e-4399-bb83-6159c027e6e3 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:14.455180 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cc84f-346e-4399-bb83-6159c027e6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:14.474986 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-473cc84f-346e-4399-bb83-6159c027e6e3 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:14.476201 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-473cc84f-346e-4399-bb83-6159c027e6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:14.536392 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1353/2706] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:41:14.545997 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68634d89-69bb-4af5-8aef-eaef6077f0f8 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:14.547444 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-68634d89-69bb-4af5-8aef-eaef6077f0f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:14.605157 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68634d89-69bb-4af5-8aef-eaef6077f0f8 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:14.608767 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f441d-842f-462e-a4f8-620890482946 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:14.609371 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f441d-842f-462e-a4f8-620890482946 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:14.609654 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f441d-842f-462e-a4f8-620890482946 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:14.609929 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f441d-842f-462e-a4f8-620890482946 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:14.619473 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f441d-842f-462e-a4f8-620890482946 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:14.620886 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f441d-842f-462e-a4f8-620890482946 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:14.630091 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1354/2707] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:14.890615 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473cc84f-346e-4399-bb83-6159c027e6e3 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:14.892090 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-473cc84f-346e-4399-bb83-6159c027e6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:14.903969 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473cc84f-346e-4399-bb83-6159c027e6e3 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:14.908779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824cbe0-c01b-4d30-a56c-7265598a0f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:14.909418 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824cbe0-c01b-4d30-a56c-7265598a0f2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:14.909698 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824cbe0-c01b-4d30-a56c-7265598a0f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:14.909969 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824cbe0-c01b-4d30-a56c-7265598a0f2f admin admin] PATH_INFO: `/v3/projects/2fe8a2a279f044af808d7814e29d0531` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:14.911160 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8824cbe0-c01b-4d30-a56c-7265598a0f2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fe8a2a279f044af808d7814e29d0531)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:14.913197 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8824cbe0-c01b-4d30-a56c-7265598a0f2f admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:14.945654 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:14.963757 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-8824cbe0-c01b-4d30-a56c-7265598a0f2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fe8a2a279f044af808d7814e29d0531', 'request_id': 'req-8824cbe0-c01b-4d30-a56c-7265598a0f2f'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:14.970347 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-8824cbe0-c01b-4d30-a56c-7265598a0f2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fe8a2a279f044af808d7814e29d0531', 'request_id': 'req-8824cbe0-c01b-4d30-a56c-7265598a0f2f'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:14.976859 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-8824cbe0-c01b-4d30-a56c-7265598a0f2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fe8a2a279f044af808d7814e29d0531', 'request_id': 'req-8824cbe0-c01b-4d30-a56c-7265598a0f2f'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:14.987443 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-8824cbe0-c01b-4d30-a56c-7265598a0f2f admin admin] The token cache is being invalidate because project 2fe8a2a279f044af808d7814e29d0531 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:41:14.987703 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-8824cbe0-c01b-4d30-a56c-7265598a0f2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:14.988423 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-8824cbe0-c01b-4d30-a56c-7265598a0f2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:14.990223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1354/2708] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:41:14 2022] DELETE /identity/v3/projects/2fe8a2a279f044af808d7814e29d0531 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:41:14.994762 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f441d-842f-462e-a4f8-620890482946 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:15.004189 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f441d-842f-462e-a4f8-620890482946 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:15.058776 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998d5443-0f3f-41db-b21a-02107895fa43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:15.059398 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d5443-0f3f-41db-b21a-02107895fa43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:15.059676 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d5443-0f3f-41db-b21a-02107895fa43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:15.059949 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d5443-0f3f-41db-b21a-02107895fa43 admin admin] PATH_INFO: `/v3/users/db7a9ffefc5047c7907ac941ceaadcbc` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:15.061053 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998d5443-0f3f-41db-b21a-02107895fa43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db7a9ffefc5047c7907ac941ceaadcbc)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:15.062976 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998d5443-0f3f-41db-b21a-02107895fa43 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:15.107632 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-998d5443-0f3f-41db-b21a-02107895fa43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db7a9ffefc5047c7907ac941ceaadcbc'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:15.114559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-998d5443-0f3f-41db-b21a-02107895fa43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db7a9ffefc5047c7907ac941ceaadcbc'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:15.115180 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-998d5443-0f3f-41db-b21a-02107895fa43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db7a9ffefc5047c7907ac941ceaadcbc'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:15.120162 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:15.129268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-998d5443-0f3f-41db-b21a-02107895fa43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db7a9ffefc5047c7907ac941ceaadcbc'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:15.130101 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-998d5443-0f3f-41db-b21a-02107895fa43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db7a9ffefc5047c7907ac941ceaadcbc'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:15.142889 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1355/2709] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:41:14 2022] DELETE /identity/v3/users/db7a9ffefc5047c7907ac941ceaadcbc => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:15.449058 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824cbe0-c01b-4d30-a56c-7265598a0f2f admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:15.450753 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-8824cbe0-c01b-4d30-a56c-7265598a0f2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:15.465161 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824cbe0-c01b-4d30-a56c-7265598a0f2f admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:15.528252 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e50d46-cfb3-4a2f-bb75-b0bd9ab34331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:15.528901 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e50d46-cfb3-4a2f-bb75-b0bd9ab34331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:15.529163 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e50d46-cfb3-4a2f-bb75-b0bd9ab34331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:15.529440 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e50d46-cfb3-4a2f-bb75-b0bd9ab34331 admin admin] PATH_INFO: `/v3/projects/1d0c8492cd9441a892bf0ddd5fa69d62` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:15.530577 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e50d46-cfb3-4a2f-bb75-b0bd9ab34331 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d0c8492cd9441a892bf0ddd5fa69d62)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:15.532483 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e50d46-cfb3-4a2f-bb75-b0bd9ab34331 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:15.578357 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-48e50d46-cfb3-4a2f-bb75-b0bd9ab34331 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d0c8492cd9441a892bf0ddd5fa69d62', 'request_id': 'req-48e50d46-cfb3-4a2f-bb75-b0bd9ab34331'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:15.585925 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-48e50d46-cfb3-4a2f-bb75-b0bd9ab34331 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d0c8492cd9441a892bf0ddd5fa69d62', 'request_id': 'req-48e50d46-cfb3-4a2f-bb75-b0bd9ab34331'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:15.592323 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-48e50d46-cfb3-4a2f-bb75-b0bd9ab34331 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d0c8492cd9441a892bf0ddd5fa69d62', 'request_id': 'req-48e50d46-cfb3-4a2f-bb75-b0bd9ab34331'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:15.602757 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-48e50d46-cfb3-4a2f-bb75-b0bd9ab34331 admin admin] The token cache is being invalidate because project 1d0c8492cd9441a892bf0ddd5fa69d62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:41:15.603049 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-48e50d46-cfb3-4a2f-bb75-b0bd9ab34331 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:15.603680 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-48e50d46-cfb3-4a2f-bb75-b0bd9ab34331 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:15.605150 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1355/2710] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:41:15 2022] DELETE /identity/v3/projects/1d0c8492cd9441a892bf0ddd5fa69d62 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:15.610003 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e12b9-d078-4c5a-b940-c1fd147451bb None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:15.610557 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e12b9-d078-4c5a-b940-c1fd147451bb None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:15.610557 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e12b9-d078-4c5a-b940-c1fd147451bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:15.631040 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-023e12b9-d078-4c5a-b940-c1fd147451bb None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:15.632225 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-023e12b9-d078-4c5a-b940-c1fd147451bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:15.689410 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1356/2711] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:15.698621 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c31d9b-cc1c-4c54-926b-c8ec7595d835 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:15.698913 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c31d9b-cc1c-4c54-926b-c8ec7595d835 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:15.699193 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c31d9b-cc1c-4c54-926b-c8ec7595d835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:15.719514 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-22c31d9b-cc1c-4c54-926b-c8ec7595d835 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:15.720723 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-22c31d9b-cc1c-4c54-926b-c8ec7595d835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:15.729450 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1356/2712] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:41:15.733833 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e12b9-d078-4c5a-b940-c1fd147451bb None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:15.743350 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e12b9-d078-4c5a-b940-c1fd147451bb None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:15.748533 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82ef6b-d020-481f-bbc7-d81e2effc9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:15.749123 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82ef6b-d020-481f-bbc7-d81e2effc9e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:15.749410 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82ef6b-d020-481f-bbc7-d81e2effc9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:15.749670 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82ef6b-d020-481f-bbc7-d81e2effc9e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:15.750371 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82ef6b-d020-481f-bbc7-d81e2effc9e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:41:15.750723 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82ef6b-d020-481f-bbc7-d81e2effc9e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:15.752318 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82ef6b-d020-481f-bbc7-d81e2effc9e5 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:15.773388 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1357/2713] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:41:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:41:15.778159 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c31d9b-cc1c-4c54-926b-c8ec7595d835 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:15.797663 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c31d9b-cc1c-4c54-926b-c8ec7595d835 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:15.803725 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cfc1d1-bab3-437f-8356-e314b5ec24cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:15.804331 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfc1d1-bab3-437f-8356-e314b5ec24cf admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:15.804610 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfc1d1-bab3-437f-8356-e314b5ec24cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:15.804879 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfc1d1-bab3-437f-8356-e314b5ec24cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:15.805550 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cfc1d1-bab3-437f-8356-e314b5ec24cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:15.808610 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cfc1d1-bab3-437f-8356-e314b5ec24cf admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:15.843916 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1357/2714] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:41:15 2022] POST /identity/v3/projects => generated 381 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:15.845108 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82ef6b-d020-481f-bbc7-d81e2effc9e5 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:15.856313 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82ef6b-d020-481f-bbc7-d81e2effc9e5 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:15.910880 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5db514-4381-4153-855d-8af415abd61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:15.911491 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5db514-4381-4153-855d-8af415abd61b admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:15.911769 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5db514-4381-4153-855d-8af415abd61b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:15.912037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5db514-4381-4153-855d-8af415abd61b admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:15.912651 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5db514-4381-4153-855d-8af415abd61b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:15.914580 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5db514-4381-4153-855d-8af415abd61b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:15.941252 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1358/2715] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:41:15 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:15.942501 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cfc1d1-bab3-437f-8356-e314b5ec24cf admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:15.952891 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cfc1d1-bab3-437f-8356-e314b5ec24cf admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:15.957538 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c65034-e24d-44be-be7a-317acf95d735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:15.958100 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c65034-e24d-44be-be7a-317acf95d735 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:15.958502 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c65034-e24d-44be-be7a-317acf95d735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:15.958695 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c65034-e24d-44be-be7a-317acf95d735 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:15.959437 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c65034-e24d-44be-be7a-317acf95d735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:15.961172 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c65034-e24d-44be-be7a-317acf95d735 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:15.971945 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1358/2716] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:15.976176 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5db514-4381-4153-855d-8af415abd61b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:15.987229 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5db514-4381-4153-855d-8af415abd61b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:15.992307 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d90e1-4711-4f4a-baed-1800418dbef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:15.992911 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d90e1-4711-4f4a-baed-1800418dbef3 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:15.993194 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d90e1-4711-4f4a-baed-1800418dbef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:15.993474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d90e1-4711-4f4a-baed-1800418dbef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:15.994757 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d90e1-4711-4f4a-baed-1800418dbef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:15.996332 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d90e1-4711-4f4a-baed-1800418dbef3 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:16.007578 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1359/2717] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:16.011724 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c65034-e24d-44be-be7a-317acf95d735 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:16.018262 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:16.022133 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c65034-e24d-44be-be7a-317acf95d735 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:16.028450 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90789b66-2ee2-4efd-a4b9-86e7e34fbc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:16.029097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90789b66-2ee2-4efd-a4b9-86e7e34fbc8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:16.029381 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90789b66-2ee2-4efd-a4b9-86e7e34fbc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:16.029661 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90789b66-2ee2-4efd-a4b9-86e7e34fbc8a admin admin] PATH_INFO: `/v3/projects/9177ba29f82a4149afcbcbe653a557f0/users/5daae8ace45f4b5a80e2cfd9cd30f8b6/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:16.045607 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90789b66-2ee2-4efd-a4b9-86e7e34fbc8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9177ba29f82a4149afcbcbe653a557f0, user_id=5daae8ace45f4b5a80e2cfd9cd30f8b6, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:16.048334 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90789b66-2ee2-4efd-a4b9-86e7e34fbc8a admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:16.061645 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1359/2718] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:41:16 2022] PUT /identity/v3/projects/9177ba29f82a4149afcbcbe653a557f0/users/5daae8ace45f4b5a80e2cfd9cd30f8b6/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:16.072010 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732e9e9-a5d3-4c94-9cc9-9763e4355e97 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:16.072301 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732e9e9-a5d3-4c94-9cc9-9763e4355e97 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:16.072580 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732e9e9-a5d3-4c94-9cc9-9763e4355e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:16.100214 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-2732e9e9-a5d3-4c94-9cc9-9763e4355e97 None None] MFA Rules not processed for user `5daae8ace45f4b5a80e2cfd9cd30f8b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:16.101430 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-2732e9e9-a5d3-4c94-9cc9-9763e4355e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:16.132701 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:16.169802 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1360/2719] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:16.174217 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90789b66-2ee2-4efd-a4b9-86e7e34fbc8a admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:16.235167 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90789b66-2ee2-4efd-a4b9-86e7e34fbc8a admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:16.239729 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a44d30-8dcc-4b9a-9436-0a98a93cc0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:16.240327 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a44d30-8dcc-4b9a-9436-0a98a93cc0e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:16.240607 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a44d30-8dcc-4b9a-9436-0a98a93cc0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:16.240874 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a44d30-8dcc-4b9a-9436-0a98a93cc0e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:16.241507 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a44d30-8dcc-4b9a-9436-0a98a93cc0e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:16.243445 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a44d30-8dcc-4b9a-9436-0a98a93cc0e8 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:16.270107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1360/2720] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:41:16 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:16.274175 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732e9e9-a5d3-4c94-9cc9-9763e4355e97 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:16.285234 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732e9e9-a5d3-4c94-9cc9-9763e4355e97 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:16.341549 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8fde73-bc0c-49c2-a8cd-2df0f26a1c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:16.341800 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8fde73-bc0c-49c2-a8cd-2df0f26a1c2b admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:16.341950 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8fde73-bc0c-49c2-a8cd-2df0f26a1c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:16.342224 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8fde73-bc0c-49c2-a8cd-2df0f26a1c2b admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:16.342888 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8fde73-bc0c-49c2-a8cd-2df0f26a1c2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:16.344788 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8fde73-bc0c-49c2-a8cd-2df0f26a1c2b admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:16.371241 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1361/2721] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:41:16 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:16.372535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a44d30-8dcc-4b9a-9436-0a98a93cc0e8 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:16.382990 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a44d30-8dcc-4b9a-9436-0a98a93cc0e8 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:16.388111 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c8886a-a227-49da-a21d-cd93ba1ed174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:16.388708 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8886a-a227-49da-a21d-cd93ba1ed174 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:16.388994 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8886a-a227-49da-a21d-cd93ba1ed174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:16.389268 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8886a-a227-49da-a21d-cd93ba1ed174 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:16.389910 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c8886a-a227-49da-a21d-cd93ba1ed174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:16.391556 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c8886a-a227-49da-a21d-cd93ba1ed174 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:16.402646 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1361/2722] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:16.406635 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8fde73-bc0c-49c2-a8cd-2df0f26a1c2b admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:16.416690 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8fde73-bc0c-49c2-a8cd-2df0f26a1c2b admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:16.421245 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e3517-ff99-4afe-81cf-24c9138b73ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:16.421906 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e3517-ff99-4afe-81cf-24c9138b73ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:16.422189 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e3517-ff99-4afe-81cf-24c9138b73ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:16.422499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e3517-ff99-4afe-81cf-24c9138b73ea admin admin] PATH_INFO: `/v3/projects/cdeee35deb1147ddb5809a90ee8cf4ad/users/ec3bdc8385084b80b63ae89f50370a79/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:16.438352 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e3517-ff99-4afe-81cf-24c9138b73ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdeee35deb1147ddb5809a90ee8cf4ad, user_id=ec3bdc8385084b80b63ae89f50370a79, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:16.441049 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e3517-ff99-4afe-81cf-24c9138b73ea admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:16.456165 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1362/2723] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:41:16 2022] PUT /identity/v3/projects/cdeee35deb1147ddb5809a90ee8cf4ad/users/ec3bdc8385084b80b63ae89f50370a79/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:16.460244 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c8886a-a227-49da-a21d-cd93ba1ed174 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:16.471276 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c8886a-a227-49da-a21d-cd93ba1ed174 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:16.529054 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f6ba6e-191c-4e96-a92a-084bcf33c141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:16.529666 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6ba6e-191c-4e96-a92a-084bcf33c141 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:16.529966 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6ba6e-191c-4e96-a92a-084bcf33c141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:16.530223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6ba6e-191c-4e96-a92a-084bcf33c141 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:16.530895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f6ba6e-191c-4e96-a92a-084bcf33c141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:16.532490 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f6ba6e-191c-4e96-a92a-084bcf33c141 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:16.542751 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1362/2724] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:16 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:16.546896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e3517-ff99-4afe-81cf-24c9138b73ea admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:16.557190 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e3517-ff99-4afe-81cf-24c9138b73ea admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:16.561835 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca47e2-1d4b-4fe7-bf5f-9daac0724ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:16.562546 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca47e2-1d4b-4fe7-bf5f-9daac0724ae9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:16.562831 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca47e2-1d4b-4fe7-bf5f-9daac0724ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:16.563106 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca47e2-1d4b-4fe7-bf5f-9daac0724ae9 admin admin] PATH_INFO: `/v3/projects/cdeee35deb1147ddb5809a90ee8cf4ad/users/ec3bdc8385084b80b63ae89f50370a79/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:16.565865 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca47e2-1d4b-4fe7-bf5f-9daac0724ae9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdeee35deb1147ddb5809a90ee8cf4ad, user_id=ec3bdc8385084b80b63ae89f50370a79, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:16.568566 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca47e2-1d4b-4fe7-bf5f-9daac0724ae9 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:16.579875 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1363/2725] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:41:16 2022] PUT /identity/v3/projects/cdeee35deb1147ddb5809a90ee8cf4ad/users/ec3bdc8385084b80b63ae89f50370a79/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:16.591486 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef84be-522d-4763-86c6-f33ca6577f09 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:16.591776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef84be-522d-4763-86c6-f33ca6577f09 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:16.592052 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef84be-522d-4763-86c6-f33ca6577f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:16.619663 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-38ef84be-522d-4763-86c6-f33ca6577f09 None None] MFA Rules not processed for user `ec3bdc8385084b80b63ae89f50370a79`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:16.620865 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-38ef84be-522d-4763-86c6-f33ca6577f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:16.660827 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:16.690162 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1363/2726] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:16 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:16.702208 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca47e2-1d4b-4fe7-bf5f-9daac0724ae9 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:16.704051 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-05ca47e2-1d4b-4fe7-bf5f-9daac0724ae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:16.762127 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca47e2-1d4b-4fe7-bf5f-9daac0724ae9 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:16.765647 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119956e-739e-4ecd-9c83-05283428ce27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:16.766271 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119956e-739e-4ecd-9c83-05283428ce27 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:16.766562 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119956e-739e-4ecd-9c83-05283428ce27 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:16.766841 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119956e-739e-4ecd-9c83-05283428ce27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:16.776105 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1119956e-739e-4ecd-9c83-05283428ce27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:16.777459 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1119956e-739e-4ecd-9c83-05283428ce27 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:16.786998 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1364/2727] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:16 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:16.860094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef84be-522d-4763-86c6-f33ca6577f09 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:16.870254 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef84be-522d-4763-86c6-f33ca6577f09 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:16.873846 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84370daf-35e6-4bba-816b-531419b55742 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:16.874487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84370daf-35e6-4bba-816b-531419b55742 service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:16.874764 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84370daf-35e6-4bba-816b-531419b55742 service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:16.875039 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84370daf-35e6-4bba-816b-531419b55742 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:16.930062 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84370daf-35e6-4bba-816b-531419b55742 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:16.932041 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84370daf-35e6-4bba-816b-531419b55742 service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:16.940708 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1364/2728] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:16 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:17.213592 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119956e-739e-4ecd-9c83-05283428ce27 service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:17.245100 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:17.281009 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119956e-739e-4ecd-9c83-05283428ce27 service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:17.285463 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89343d1-da91-4257-91ab-4f6a84e26909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:17.286093 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89343d1-da91-4257-91ab-4f6a84e26909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:17.286415 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89343d1-da91-4257-91ab-4f6a84e26909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:17.286705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89343d1-da91-4257-91ab-4f6a84e26909 admin admin] PATH_INFO: `/v3/users/5daae8ace45f4b5a80e2cfd9cd30f8b6` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:17.287839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89343d1-da91-4257-91ab-4f6a84e26909 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5daae8ace45f4b5a80e2cfd9cd30f8b6)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:17.289731 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89343d1-da91-4257-91ab-4f6a84e26909 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:17.334693 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e89343d1-da91-4257-91ab-4f6a84e26909 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5daae8ace45f4b5a80e2cfd9cd30f8b6'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:17.342011 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e89343d1-da91-4257-91ab-4f6a84e26909 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5daae8ace45f4b5a80e2cfd9cd30f8b6'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:17.342743 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e89343d1-da91-4257-91ab-4f6a84e26909 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5daae8ace45f4b5a80e2cfd9cd30f8b6'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:17.356698 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e89343d1-da91-4257-91ab-4f6a84e26909 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5daae8ace45f4b5a80e2cfd9cd30f8b6'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:17.357398 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e89343d1-da91-4257-91ab-4f6a84e26909 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5daae8ace45f4b5a80e2cfd9cd30f8b6'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:17.370636 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1365/2729] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:41:17 2022] DELETE /identity/v3/users/5daae8ace45f4b5a80e2cfd9cd30f8b6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:17.375226 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c2cdf-6699-47e5-b20e-9dadab48813e None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:17.375516 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c2cdf-6699-47e5-b20e-9dadab48813e None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:17.375789 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c2cdf-6699-47e5-b20e-9dadab48813e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:17.394378 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-8b9c2cdf-6699-47e5-b20e-9dadab48813e None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:17.395557 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-8b9c2cdf-6699-47e5-b20e-9dadab48813e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:17.452881 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1365/2730] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:41:17.462648 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89343d1-da91-4257-91ab-4f6a84e26909 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:17.464090 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-e89343d1-da91-4257-91ab-4f6a84e26909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:17.521968 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89343d1-da91-4257-91ab-4f6a84e26909 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:17.526153 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f7c84-60ba-471c-9fbf-a342aafc00af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:17.526809 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f7c84-60ba-471c-9fbf-a342aafc00af service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:17.527095 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f7c84-60ba-471c-9fbf-a342aafc00af service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:17.527369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f7c84-60ba-471c-9fbf-a342aafc00af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:17.537233 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f7c84-60ba-471c-9fbf-a342aafc00af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:17.538630 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f7c84-60ba-471c-9fbf-a342aafc00af service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:17.550853 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1366/2731] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:17.867879 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c2cdf-6699-47e5-b20e-9dadab48813e None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:17.869637 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-8b9c2cdf-6699-47e5-b20e-9dadab48813e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:17.881557 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c2cdf-6699-47e5-b20e-9dadab48813e None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:17.886760 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fdf682-d8ac-414c-9cd3-52952b9eaaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:17.887396 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fdf682-d8ac-414c-9cd3-52952b9eaaa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:17.887680 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fdf682-d8ac-414c-9cd3-52952b9eaaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:17.887955 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fdf682-d8ac-414c-9cd3-52952b9eaaa6 admin admin] PATH_INFO: `/v3/projects/9177ba29f82a4149afcbcbe653a557f0` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:17.889088 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fdf682-d8ac-414c-9cd3-52952b9eaaa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9177ba29f82a4149afcbcbe653a557f0)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:17.891038 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fdf682-d8ac-414c-9cd3-52952b9eaaa6 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:17.938493 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-f1fdf682-d8ac-414c-9cd3-52952b9eaaa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9177ba29f82a4149afcbcbe653a557f0', 'request_id': 'req-f1fdf682-d8ac-414c-9cd3-52952b9eaaa6'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:17.946517 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-f1fdf682-d8ac-414c-9cd3-52952b9eaaa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9177ba29f82a4149afcbcbe653a557f0', 'request_id': 'req-f1fdf682-d8ac-414c-9cd3-52952b9eaaa6'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:17.953267 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-f1fdf682-d8ac-414c-9cd3-52952b9eaaa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9177ba29f82a4149afcbcbe653a557f0', 'request_id': 'req-f1fdf682-d8ac-414c-9cd3-52952b9eaaa6'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:17.959702 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:17.965455 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-f1fdf682-d8ac-414c-9cd3-52952b9eaaa6 admin admin] The token cache is being invalidate because project 9177ba29f82a4149afcbcbe653a557f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:41:17.965799 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-f1fdf682-d8ac-414c-9cd3-52952b9eaaa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:17.966477 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-f1fdf682-d8ac-414c-9cd3-52952b9eaaa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:17.968260 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1366/2732] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:41:17 2022] DELETE /identity/v3/projects/9177ba29f82a4149afcbcbe653a557f0 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:41:17.972744 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f7c84-60ba-471c-9fbf-a342aafc00af service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:17.982571 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f7c84-60ba-471c-9fbf-a342aafc00af service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:18.026662 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:18.041125 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec7b52-a80e-4a00-9545-f3f084da45f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:18.041745 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec7b52-a80e-4a00-9545-f3f084da45f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:18.042023 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec7b52-a80e-4a00-9545-f3f084da45f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:18.042327 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec7b52-a80e-4a00-9545-f3f084da45f2 admin admin] PATH_INFO: `/v3/users/ec3bdc8385084b80b63ae89f50370a79` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:18.043437 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec7b52-a80e-4a00-9545-f3f084da45f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec3bdc8385084b80b63ae89f50370a79)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:18.045331 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec7b52-a80e-4a00-9545-f3f084da45f2 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:18.090702 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-afec7b52-a80e-4a00-9545-f3f084da45f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec3bdc8385084b80b63ae89f50370a79'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:18.097782 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-afec7b52-a80e-4a00-9545-f3f084da45f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec3bdc8385084b80b63ae89f50370a79'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:18.098454 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-afec7b52-a80e-4a00-9545-f3f084da45f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec3bdc8385084b80b63ae89f50370a79'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:18.112121 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-afec7b52-a80e-4a00-9545-f3f084da45f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec3bdc8385084b80b63ae89f50370a79'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:18.112745 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-afec7b52-a80e-4a00-9545-f3f084da45f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec3bdc8385084b80b63ae89f50370a79'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:18.125080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1367/2733] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:41:17 2022] DELETE /identity/v3/users/ec3bdc8385084b80b63ae89f50370a79 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:18.387036 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fdf682-d8ac-414c-9cd3-52952b9eaaa6 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:18.388467 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-f1fdf682-d8ac-414c-9cd3-52952b9eaaa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:18.403335 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fdf682-d8ac-414c-9cd3-52952b9eaaa6 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:18.460291 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725f8f4-802f-4796-8ffc-f5a8b9cfda83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:18.460914 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725f8f4-802f-4796-8ffc-f5a8b9cfda83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:18.461190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725f8f4-802f-4796-8ffc-f5a8b9cfda83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:18.461458 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725f8f4-802f-4796-8ffc-f5a8b9cfda83 admin admin] PATH_INFO: `/v3/projects/cdeee35deb1147ddb5809a90ee8cf4ad` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:18.463084 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725f8f4-802f-4796-8ffc-f5a8b9cfda83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdeee35deb1147ddb5809a90ee8cf4ad)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:18.464974 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725f8f4-802f-4796-8ffc-f5a8b9cfda83 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:18.511846 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-d725f8f4-802f-4796-8ffc-f5a8b9cfda83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdeee35deb1147ddb5809a90ee8cf4ad', 'request_id': 'req-d725f8f4-802f-4796-8ffc-f5a8b9cfda83'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:18.518768 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-d725f8f4-802f-4796-8ffc-f5a8b9cfda83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdeee35deb1147ddb5809a90ee8cf4ad', 'request_id': 'req-d725f8f4-802f-4796-8ffc-f5a8b9cfda83'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:18.525550 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-d725f8f4-802f-4796-8ffc-f5a8b9cfda83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdeee35deb1147ddb5809a90ee8cf4ad', 'request_id': 'req-d725f8f4-802f-4796-8ffc-f5a8b9cfda83'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:18.535765 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-d725f8f4-802f-4796-8ffc-f5a8b9cfda83 admin admin] The token cache is being invalidate because project cdeee35deb1147ddb5809a90ee8cf4ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:41:18.536056 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-d725f8f4-802f-4796-8ffc-f5a8b9cfda83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:18.536684 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-d725f8f4-802f-4796-8ffc-f5a8b9cfda83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:18.538157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1367/2734] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:41:18 2022] DELETE /identity/v3/projects/cdeee35deb1147ddb5809a90ee8cf4ad => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:18.543671 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c540a5-b925-4764-9fdd-db0ddcc1aa74 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:18.543963 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c540a5-b925-4764-9fdd-db0ddcc1aa74 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:18.544244 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c540a5-b925-4764-9fdd-db0ddcc1aa74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:18.563934 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-18c540a5-b925-4764-9fdd-db0ddcc1aa74 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:18.565099 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-18c540a5-b925-4764-9fdd-db0ddcc1aa74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:18.624778 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1368/2735] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:18.633119 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0734e6-8acd-4e35-812b-c9beec0251a8 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:18.633413 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0734e6-8acd-4e35-812b-c9beec0251a8 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:18.633693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0734e6-8acd-4e35-812b-c9beec0251a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:18.654572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-ac0734e6-8acd-4e35-812b-c9beec0251a8 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:18.655767 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-ac0734e6-8acd-4e35-812b-c9beec0251a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:18.663503 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1368/2736] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:18.667995 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c540a5-b925-4764-9fdd-db0ddcc1aa74 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:18.677829 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c540a5-b925-4764-9fdd-db0ddcc1aa74 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:18.682560 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43bc55-66aa-41a5-a0a2-a3dfa1ff8db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:18.683158 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43bc55-66aa-41a5-a0a2-a3dfa1ff8db2 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:18.683441 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43bc55-66aa-41a5-a0a2-a3dfa1ff8db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:18.683715 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43bc55-66aa-41a5-a0a2-a3dfa1ff8db2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:18.684356 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d43bc55-66aa-41a5-a0a2-a3dfa1ff8db2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:41:18.684656 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d43bc55-66aa-41a5-a0a2-a3dfa1ff8db2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:18.686322 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d43bc55-66aa-41a5-a0a2-a3dfa1ff8db2 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:18.698257 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1369/2737] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:41:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:41:18.702701 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0734e6-8acd-4e35-812b-c9beec0251a8 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:18.711866 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0734e6-8acd-4e35-812b-c9beec0251a8 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:18.716981 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a6deff-479a-4c92-a879-672da93333b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:18.717576 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6deff-479a-4c92-a879-672da93333b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:18.717853 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6deff-479a-4c92-a879-672da93333b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:18.718125 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6deff-479a-4c92-a879-672da93333b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:18.718788 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a6deff-479a-4c92-a879-672da93333b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:18.720691 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a6deff-479a-4c92-a879-672da93333b1 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:18.747513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1369/2738] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:41:18 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:18.751686 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43bc55-66aa-41a5-a0a2-a3dfa1ff8db2 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:18.761995 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43bc55-66aa-41a5-a0a2-a3dfa1ff8db2 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:18.811056 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:18.819432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1eb1a-28c3-4406-91f8-3d57ff304279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:18.820211 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1eb1a-28c3-4406-91f8-3d57ff304279 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:18.820725 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1eb1a-28c3-4406-91f8-3d57ff304279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:18.821320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1eb1a-28c3-4406-91f8-3d57ff304279 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:18.822810 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1eb1a-28c3-4406-91f8-3d57ff304279 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:18.824726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1eb1a-28c3-4406-91f8-3d57ff304279 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:18.847270 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1370/2739] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:41:18 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:18.851807 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a6deff-479a-4c92-a879-672da93333b1 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:18.861971 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a6deff-479a-4c92-a879-672da93333b1 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:18.866614 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42384720-997a-4bbb-afe9-c3872465934d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:18.867215 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42384720-997a-4bbb-afe9-c3872465934d admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:18.867500 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42384720-997a-4bbb-afe9-c3872465934d admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:18.867776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42384720-997a-4bbb-afe9-c3872465934d admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:18.868420 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42384720-997a-4bbb-afe9-c3872465934d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:18.870146 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42384720-997a-4bbb-afe9-c3872465934d admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:18.880829 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1370/2740] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:18.885521 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1eb1a-28c3-4406-91f8-3d57ff304279 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:18.895425 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1eb1a-28c3-4406-91f8-3d57ff304279 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:18.900645 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73cfe97-18f1-4dc5-b32a-a4890e845651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:18.901243 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73cfe97-18f1-4dc5-b32a-a4890e845651 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:18.901522 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73cfe97-18f1-4dc5-b32a-a4890e845651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:18.901827 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73cfe97-18f1-4dc5-b32a-a4890e845651 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:18.902492 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73cfe97-18f1-4dc5-b32a-a4890e845651 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:18.904064 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73cfe97-18f1-4dc5-b32a-a4890e845651 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:18.915097 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1371/2741] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:18.919305 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42384720-997a-4bbb-afe9-c3872465934d admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:18.929968 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42384720-997a-4bbb-afe9-c3872465934d admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:18.934312 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab70aa-3460-465e-8c82-875bcef95642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:18.934978 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab70aa-3460-465e-8c82-875bcef95642 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:18.935263 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab70aa-3460-465e-8c82-875bcef95642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:18.935540 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab70aa-3460-465e-8c82-875bcef95642 admin admin] PATH_INFO: `/v3/projects/800f5a26cfcc46aeac51f287032b4222/users/b3a103c148954c0e9e101a1d0f7ff8c9/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:18.952228 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ab70aa-3460-465e-8c82-875bcef95642 admin admin] RBAC: Authorizing `identity:create_grant(project_id=800f5a26cfcc46aeac51f287032b4222, user_id=b3a103c148954c0e9e101a1d0f7ff8c9, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:18.955093 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ab70aa-3460-465e-8c82-875bcef95642 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:18.969192 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1371/2742] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:41:18 2022] PUT /identity/v3/projects/800f5a26cfcc46aeac51f287032b4222/users/b3a103c148954c0e9e101a1d0f7ff8c9/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:41:18.979839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bed3d1-8427-4c73-80b8-82931381151b None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:18.980355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bed3d1-8427-4c73-80b8-82931381151b None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:18.980484 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bed3d1-8427-4c73-80b8-82931381151b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:19.009034 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-14bed3d1-8427-4c73-80b8-82931381151b None None] MFA Rules not processed for user `b3a103c148954c0e9e101a1d0f7ff8c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:19.010853 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-14bed3d1-8427-4c73-80b8-82931381151b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:19.076544 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1372/2743] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:19.087955 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab70aa-3460-465e-8c82-875bcef95642 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:19.089758 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-c3ab70aa-3460-465e-8c82-875bcef95642 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:19.117951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:19.150532 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab70aa-3460-465e-8c82-875bcef95642 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:19.154079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e7210d-a88d-411a-8dd4-a9363315ec31 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b09f6d550c9142bf956b225bf5172e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:19.154754 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7210d-a88d-411a-8dd4-a9363315ec31 service glance] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:19.155034 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7210d-a88d-411a-8dd4-a9363315ec31 service glance] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:19.155309 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7210d-a88d-411a-8dd4-a9363315ec31 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:19.164094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e7210d-a88d-411a-8dd4-a9363315ec31 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:19.166033 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e7210d-a88d-411a-8dd4-a9363315ec31 service glance] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:19.176444 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1372/2744] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:29.073530 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=41,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:29.172121 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=15 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:31.482021 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bed3d1-8427-4c73-80b8-82931381151b None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:31.483487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-14bed3d1-8427-4c73-80b8-82931381151b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:31.543567 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bed3d1-8427-4c73-80b8-82931381151b None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:31.547250 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd04b0-4078-4d31-83f8-f0ded3474979 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:31.547857 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd04b0-4078-4d31-83f8-f0ded3474979 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:31.548134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd04b0-4078-4d31-83f8-f0ded3474979 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:31.548404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd04b0-4078-4d31-83f8-f0ded3474979 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:31.550099 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ccfd04b0-4078-4d31-83f8-f0ded3474979 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:31.615663 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd04b0-4078-4d31-83f8-f0ded3474979 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:31.617096 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd04b0-4078-4d31-83f8-f0ded3474979 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:31.627163 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1373/2745] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:31 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:32.084123 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e7210d-a88d-411a-8dd4-a9363315ec31 service glance] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:32.085713 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-b0e7210d-a88d-411a-8dd4-a9363315ec31 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:32.098781 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e7210d-a88d-411a-8dd4-a9363315ec31 service glance] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:32.104342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de68ab5b-a2e6-4d4f-9cf2-adc09fad4030 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:32.105555 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de68ab5b-a2e6-4d4f-9cf2-adc09fad4030 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:32.105840 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de68ab5b-a2e6-4d4f-9cf2-adc09fad4030 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:32.106113 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de68ab5b-a2e6-4d4f-9cf2-adc09fad4030 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:32.108402 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-de68ab5b-a2e6-4d4f-9cf2-adc09fad4030 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:32.174058 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de68ab5b-a2e6-4d4f-9cf2-adc09fad4030 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:32.175479 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de68ab5b-a2e6-4d4f-9cf2-adc09fad4030 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:32.185349 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1373/2746] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:35.767336 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd04b0-4078-4d31-83f8-f0ded3474979 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:35.771898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ccfd04b0-4078-4d31-83f8-f0ded3474979 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:35.850768 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd04b0-4078-4d31-83f8-f0ded3474979 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:35.858306 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d23866-fdd7-4544-b2fa-8561e28b3b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:35.859896 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d23866-fdd7-4544-b2fa-8561e28b3b0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:35.860388 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d23866-fdd7-4544-b2fa-8561e28b3b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:35.860868 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d23866-fdd7-4544-b2fa-8561e28b3b0c admin admin] PATH_INFO: `/v3/users/8388303f9098497bb559b3bcb59fab52` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:35.862457 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d23866-fdd7-4544-b2fa-8561e28b3b0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8388303f9098497bb559b3bcb59fab52)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:35.866773 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d23866-fdd7-4544-b2fa-8561e28b3b0c admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:35.948659 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e0d23866-fdd7-4544-b2fa-8561e28b3b0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8388303f9098497bb559b3bcb59fab52'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:35.958595 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e0d23866-fdd7-4544-b2fa-8561e28b3b0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8388303f9098497bb559b3bcb59fab52'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:35.959568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e0d23866-fdd7-4544-b2fa-8561e28b3b0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8388303f9098497bb559b3bcb59fab52'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:35.977804 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e0d23866-fdd7-4544-b2fa-8561e28b3b0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8388303f9098497bb559b3bcb59fab52'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:35.978764 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e0d23866-fdd7-4544-b2fa-8561e28b3b0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8388303f9098497bb559b3bcb59fab52'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:35.997949 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1374/2747] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:41:35 2022] DELETE /identity/v3/users/8388303f9098497bb559b3bcb59fab52 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:36.175785 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de68ab5b-a2e6-4d4f-9cf2-adc09fad4030 service neutron] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:36.187262 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de68ab5b-a2e6-4d4f-9cf2-adc09fad4030 service neutron] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:36.257677 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af986229-b0d6-4b8e-a433-c251ed3582fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:36.258812 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af986229-b0d6-4b8e-a433-c251ed3582fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:36.259045 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af986229-b0d6-4b8e-a433-c251ed3582fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:36.259319 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af986229-b0d6-4b8e-a433-c251ed3582fe admin admin] PATH_INFO: `/v3/projects/cfb0b630e22745e8af15fb02c71800c3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:36.260997 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af986229-b0d6-4b8e-a433-c251ed3582fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfb0b630e22745e8af15fb02c71800c3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:36.262489 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af986229-b0d6-4b8e-a433-c251ed3582fe admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:36.319422 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-af986229-b0d6-4b8e-a433-c251ed3582fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfb0b630e22745e8af15fb02c71800c3', 'request_id': 'req-af986229-b0d6-4b8e-a433-c251ed3582fe'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:36.330798 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-af986229-b0d6-4b8e-a433-c251ed3582fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfb0b630e22745e8af15fb02c71800c3', 'request_id': 'req-af986229-b0d6-4b8e-a433-c251ed3582fe'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:36.339760 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-af986229-b0d6-4b8e-a433-c251ed3582fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfb0b630e22745e8af15fb02c71800c3', 'request_id': 'req-af986229-b0d6-4b8e-a433-c251ed3582fe'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:36.356782 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-af986229-b0d6-4b8e-a433-c251ed3582fe admin admin] The token cache is being invalidate because project cfb0b630e22745e8af15fb02c71800c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:41:36.357608 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-af986229-b0d6-4b8e-a433-c251ed3582fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:36.358371 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-af986229-b0d6-4b8e-a433-c251ed3582fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:36.360278 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1374/2748] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:41:36 2022] DELETE /identity/v3/projects/cfb0b630e22745e8af15fb02c71800c3 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:36.367694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4875d-cb17-4c7d-99ea-f905cb7d2e01 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:36.367694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4875d-cb17-4c7d-99ea-f905cb7d2e01 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:36.367694 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4875d-cb17-4c7d-99ea-f905cb7d2e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:36.386068 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-e5c4875d-cb17-4c7d-99ea-f905cb7d2e01 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:36.387312 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-e5c4875d-cb17-4c7d-99ea-f905cb7d2e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:36.404630 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:36.447194 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1375/2749] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:36.454905 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797b30b-7fa9-4313-9249-ee8e43bf87a3 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:36.455198 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797b30b-7fa9-4313-9249-ee8e43bf87a3 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:36.455478 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797b30b-7fa9-4313-9249-ee8e43bf87a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:36.475414 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-f797b30b-7fa9-4313-9249-ee8e43bf87a3 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:36.476864 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-f797b30b-7fa9-4313-9249-ee8e43bf87a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:36.485219 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1375/2750] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:36.489336 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4875d-cb17-4c7d-99ea-f905cb7d2e01 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:36.498890 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4875d-cb17-4c7d-99ea-f905cb7d2e01 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:36.503402 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a06bd-1937-4a2b-b587-da77e7dc97b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:36.503990 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a06bd-1937-4a2b-b587-da77e7dc97b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:36.504264 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a06bd-1937-4a2b-b587-da77e7dc97b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:36.504530 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a06bd-1937-4a2b-b587-da77e7dc97b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:36.505158 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5a06bd-1937-4a2b-b587-da77e7dc97b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:41:36.505453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5a06bd-1937-4a2b-b587-da77e7dc97b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:36.507122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5a06bd-1937-4a2b-b587-da77e7dc97b5 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:36.518303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1376/2751] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:41:36.522434 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f797b30b-7fa9-4313-9249-ee8e43bf87a3 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:36.532111 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f797b30b-7fa9-4313-9249-ee8e43bf87a3 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:36.536784 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33a49e-e312-453b-a56c-42e304de40c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:36.537408 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33a49e-e312-453b-a56c-42e304de40c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:36.537692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33a49e-e312-453b-a56c-42e304de40c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:36.537967 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33a49e-e312-453b-a56c-42e304de40c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:36.538719 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af33a49e-e312-453b-a56c-42e304de40c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:36.541184 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af33a49e-e312-453b-a56c-42e304de40c3 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:36.557437 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:36.571036 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1376/2752] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:41:36 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:36.575275 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a06bd-1937-4a2b-b587-da77e7dc97b5 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:36.585975 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a06bd-1937-4a2b-b587-da77e7dc97b5 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:36.647489 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f1a01-419b-4550-92a4-525c66a840c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:36.648110 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f1a01-419b-4550-92a4-525c66a840c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:36.648395 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f1a01-419b-4550-92a4-525c66a840c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:36.648667 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f1a01-419b-4550-92a4-525c66a840c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:36.649875 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111f1a01-419b-4550-92a4-525c66a840c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:36.651835 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111f1a01-419b-4550-92a4-525c66a840c1 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:36.674047 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1377/2753] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:41:36 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:36.678229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33a49e-e312-453b-a56c-42e304de40c3 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:36.688880 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33a49e-e312-453b-a56c-42e304de40c3 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:36.695331 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc847b8-fb2e-4799-809b-b61871757534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:36.696535 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc847b8-fb2e-4799-809b-b61871757534 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:36.696831 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc847b8-fb2e-4799-809b-b61871757534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:36.697174 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc847b8-fb2e-4799-809b-b61871757534 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:36.697896 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc847b8-fb2e-4799-809b-b61871757534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:36.699680 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc847b8-fb2e-4799-809b-b61871757534 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:36.714890 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1377/2754] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:36.716045 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f1a01-419b-4550-92a4-525c66a840c1 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:36.726441 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f1a01-419b-4550-92a4-525c66a840c1 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:36.731172 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3345cc42-eb5e-44d1-9e77-c87320aff28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:36.731793 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345cc42-eb5e-44d1-9e77-c87320aff28c admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:36.732075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345cc42-eb5e-44d1-9e77-c87320aff28c admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:36.732350 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345cc42-eb5e-44d1-9e77-c87320aff28c admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:36.732998 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3345cc42-eb5e-44d1-9e77-c87320aff28c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:36.734637 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3345cc42-eb5e-44d1-9e77-c87320aff28c admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:36.746391 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1378/2755] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:36.750304 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc847b8-fb2e-4799-809b-b61871757534 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:36.761183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc847b8-fb2e-4799-809b-b61871757534 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:36.766656 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0054c2d2-131f-40f4-9ac3-c34352573f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:36.767375 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054c2d2-131f-40f4-9ac3-c34352573f18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:36.767725 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054c2d2-131f-40f4-9ac3-c34352573f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:36.768071 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054c2d2-131f-40f4-9ac3-c34352573f18 admin admin] PATH_INFO: `/v3/projects/9b9aa20e1ffb453d9ee13dd6ae2c25ef/users/a237b400429f4200909581e3c5e0cbfb/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:36.784544 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0054c2d2-131f-40f4-9ac3-c34352573f18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b9aa20e1ffb453d9ee13dd6ae2c25ef, user_id=a237b400429f4200909581e3c5e0cbfb, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:36.787663 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0054c2d2-131f-40f4-9ac3-c34352573f18 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:36.801274 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1378/2756] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:41:36 2022] PUT /identity/v3/projects/9b9aa20e1ffb453d9ee13dd6ae2c25ef/users/a237b400429f4200909581e3c5e0cbfb/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:36.807213 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94085bf-f971-44bc-98e7-46c07a42fb91 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:36.807784 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94085bf-f971-44bc-98e7-46c07a42fb91 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:36.807784 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94085bf-f971-44bc-98e7-46c07a42fb91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:36.831378 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-a94085bf-f971-44bc-98e7-46c07a42fb91 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:36.832830 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a94085bf-f971-44bc-98e7-46c07a42fb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:36.903994 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1379/2757] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:36.913628 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0054c2d2-131f-40f4-9ac3-c34352573f18 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:36.915433 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-0054c2d2-131f-40f4-9ac3-c34352573f18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:36.986374 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0054c2d2-131f-40f4-9ac3-c34352573f18 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:36.990623 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794a93d-1cdb-44f3-b836-e60508ff0c36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:36.991244 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794a93d-1cdb-44f3-b836-e60508ff0c36 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:36.991526 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794a93d-1cdb-44f3-b836-e60508ff0c36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:36.991935 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794a93d-1cdb-44f3-b836-e60508ff0c36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:37.003847 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794a93d-1cdb-44f3-b836-e60508ff0c36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:37.005460 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794a93d-1cdb-44f3-b836-e60508ff0c36 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:37.016037 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1379/2758] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:38.028017 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94085bf-f971-44bc-98e7-46c07a42fb91 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:38.029590 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a94085bf-f971-44bc-98e7-46c07a42fb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:38.042203 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94085bf-f971-44bc-98e7-46c07a42fb91 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:38.048406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16121d6d-1e0a-4b71-bed4-577531478d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:38.049141 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16121d6d-1e0a-4b71-bed4-577531478d49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:38.049506 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16121d6d-1e0a-4b71-bed4-577531478d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:38.049850 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16121d6d-1e0a-4b71-bed4-577531478d49 admin admin] PATH_INFO: `/v3/users/b3a103c148954c0e9e101a1d0f7ff8c9` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:38.051081 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16121d6d-1e0a-4b71-bed4-577531478d49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3a103c148954c0e9e101a1d0f7ff8c9)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:38.052995 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16121d6d-1e0a-4b71-bed4-577531478d49 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:38.078078 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:38.116472 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-16121d6d-1e0a-4b71-bed4-577531478d49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3a103c148954c0e9e101a1d0f7ff8c9'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:38.125536 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-16121d6d-1e0a-4b71-bed4-577531478d49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3a103c148954c0e9e101a1d0f7ff8c9'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:38.126384 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-16121d6d-1e0a-4b71-bed4-577531478d49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3a103c148954c0e9e101a1d0f7ff8c9'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:38.141729 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-16121d6d-1e0a-4b71-bed4-577531478d49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3a103c148954c0e9e101a1d0f7ff8c9'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:38.142505 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-16121d6d-1e0a-4b71-bed4-577531478d49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3a103c148954c0e9e101a1d0f7ff8c9'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:38.161908 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1380/2759] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:41:38 2022] DELETE /identity/v3/users/b3a103c148954c0e9e101a1d0f7ff8c9 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:38.163229 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd4e52-0bb8-42fb-a9e2-9e9805443cfb None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:38.163718 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd4e52-0bb8-42fb-a9e2-9e9805443cfb None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:38.163818 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd4e52-0bb8-42fb-a9e2-9e9805443cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:38.183834 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-a8dd4e52-0bb8-42fb-a9e2-9e9805443cfb None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:38.185067 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-a8dd4e52-0bb8-42fb-a9e2-9e9805443cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:38.251196 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1380/2760] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:41:38.261702 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16121d6d-1e0a-4b71-bed4-577531478d49 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:38.263539 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-16121d6d-1e0a-4b71-bed4-577531478d49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:38.346558 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16121d6d-1e0a-4b71-bed4-577531478d49 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:38.351873 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1baf8-90c3-49a9-b51e-aca0c8460fa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:38.352866 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1baf8-90c3-49a9-b51e-aca0c8460fa5 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:38.353312 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1baf8-90c3-49a9-b51e-aca0c8460fa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:38.353747 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1baf8-90c3-49a9-b51e-aca0c8460fa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:38.364874 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af1baf8-90c3-49a9-b51e-aca0c8460fa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:38.366721 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af1baf8-90c3-49a9-b51e-aca0c8460fa5 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:38.377792 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1381/2761] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:41:38.865143 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd4e52-0bb8-42fb-a9e2-9e9805443cfb None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:38.866641 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-a8dd4e52-0bb8-42fb-a9e2-9e9805443cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:38.882390 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd4e52-0bb8-42fb-a9e2-9e9805443cfb None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:38.895677 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3422686-09cb-45e0-9828-81c04116ed98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:38.896794 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3422686-09cb-45e0-9828-81c04116ed98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:38.897094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3422686-09cb-45e0-9828-81c04116ed98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:38.897370 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3422686-09cb-45e0-9828-81c04116ed98 admin admin] PATH_INFO: `/v3/projects/800f5a26cfcc46aeac51f287032b4222` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:38.899286 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3422686-09cb-45e0-9828-81c04116ed98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=800f5a26cfcc46aeac51f287032b4222)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:38.903140 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3422686-09cb-45e0-9828-81c04116ed98 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:38.984925 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-b3422686-09cb-45e0-9828-81c04116ed98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '800f5a26cfcc46aeac51f287032b4222', 'request_id': 'req-b3422686-09cb-45e0-9828-81c04116ed98'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:38.997647 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-b3422686-09cb-45e0-9828-81c04116ed98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '800f5a26cfcc46aeac51f287032b4222', 'request_id': 'req-b3422686-09cb-45e0-9828-81c04116ed98'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:39.001279 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:39.007262 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-b3422686-09cb-45e0-9828-81c04116ed98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '800f5a26cfcc46aeac51f287032b4222', 'request_id': 'req-b3422686-09cb-45e0-9828-81c04116ed98'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:39.021413 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-b3422686-09cb-45e0-9828-81c04116ed98 admin admin] The token cache is being invalidate because project 800f5a26cfcc46aeac51f287032b4222 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:41:39.021842 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-b3422686-09cb-45e0-9828-81c04116ed98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:39.022881 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-b3422686-09cb-45e0-9828-81c04116ed98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:39.025275 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1381/2762] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:41:38 2022] DELETE /identity/v3/projects/800f5a26cfcc46aeac51f287032b4222 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:39.029820 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e876b-8b62-4319-ab00-8ec105236aed None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:39.030348 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e876b-8b62-4319-ab00-8ec105236aed None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:39.030532 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e876b-8b62-4319-ab00-8ec105236aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:39.056050 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-4f3e876b-8b62-4319-ab00-8ec105236aed None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:39.057929 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-4f3e876b-8b62-4319-ab00-8ec105236aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:39.132229 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1382/2763] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:41:39.141066 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff9e76-536d-47d2-8e5f-5d4330e5d01a None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:39.141406 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff9e76-536d-47d2-8e5f-5d4330e5d01a None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:39.142484 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff9e76-536d-47d2-8e5f-5d4330e5d01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:39.170394 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-2dff9e76-536d-47d2-8e5f-5d4330e5d01a None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:39.172474 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-2dff9e76-536d-47d2-8e5f-5d4330e5d01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:39.187493 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e876b-8b62-4319-ab00-8ec105236aed None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:39.188035 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1382/2764] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:39.198587 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e876b-8b62-4319-ab00-8ec105236aed None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:39.205924 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008bda2-dc9c-48c8-a8ff-d1053b3b83a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:39.207375 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008bda2-dc9c-48c8-a8ff-d1053b3b83a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:39.208147 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008bda2-dc9c-48c8-a8ff-d1053b3b83a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:39.208662 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008bda2-dc9c-48c8-a8ff-d1053b3b83a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:39.209693 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c008bda2-dc9c-48c8-a8ff-d1053b3b83a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:41:39.210329 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c008bda2-dc9c-48c8-a8ff-d1053b3b83a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:39.213147 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c008bda2-dc9c-48c8-a8ff-d1053b3b83a0 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:39.232392 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1383/2765] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:41:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:41:39.236813 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff9e76-536d-47d2-8e5f-5d4330e5d01a None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:39.247993 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff9e76-536d-47d2-8e5f-5d4330e5d01a None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:39.254862 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91b5ff-30f5-4bf1-ad1f-db9ad31869b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:39.255487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91b5ff-30f5-4bf1-ad1f-db9ad31869b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:39.255767 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91b5ff-30f5-4bf1-ad1f-db9ad31869b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:39.256036 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91b5ff-30f5-4bf1-ad1f-db9ad31869b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:39.256690 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f91b5ff-30f5-4bf1-ad1f-db9ad31869b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:39.259487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f91b5ff-30f5-4bf1-ad1f-db9ad31869b4 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:39.288482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1383/2766] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:41:39 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:39.292984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008bda2-dc9c-48c8-a8ff-d1053b3b83a0 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:39.307246 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008bda2-dc9c-48c8-a8ff-d1053b3b83a0 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:39.347101 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:39.389697 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0d46f-7def-4b46-b33f-8a62ad44d936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:39.390716 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0d46f-7def-4b46-b33f-8a62ad44d936 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:39.391302 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0d46f-7def-4b46-b33f-8a62ad44d936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:39.391763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0d46f-7def-4b46-b33f-8a62ad44d936 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:39.392632 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c0d46f-7def-4b46-b33f-8a62ad44d936 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:39.395295 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c0d46f-7def-4b46-b33f-8a62ad44d936 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:39.428714 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1384/2767] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:41:39 2022] POST /identity/v3/users => generated 287 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:39.433160 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91b5ff-30f5-4bf1-ad1f-db9ad31869b4 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:39.447052 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91b5ff-30f5-4bf1-ad1f-db9ad31869b4 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:39.452464 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40973a3-c862-4426-9ad4-301c6226c879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:39.453096 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40973a3-c862-4426-9ad4-301c6226c879 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:39.453374 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40973a3-c862-4426-9ad4-301c6226c879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:39.453827 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40973a3-c862-4426-9ad4-301c6226c879 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:39.454520 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40973a3-c862-4426-9ad4-301c6226c879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:39.456853 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40973a3-c862-4426-9ad4-301c6226c879 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:39.467992 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1384/2768] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:39.472387 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0d46f-7def-4b46-b33f-8a62ad44d936 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:39.488184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0d46f-7def-4b46-b33f-8a62ad44d936 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:39.495379 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891489c7-d954-4743-98c4-eff791087010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:39.496208 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891489c7-d954-4743-98c4-eff791087010 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:39.496739 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891489c7-d954-4743-98c4-eff791087010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:39.497305 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891489c7-d954-4743-98c4-eff791087010 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:39.498710 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891489c7-d954-4743-98c4-eff791087010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:39.500527 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891489c7-d954-4743-98c4-eff791087010 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:39.513926 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1385/2769] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:39 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:41:39.518298 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40973a3-c862-4426-9ad4-301c6226c879 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:39.532158 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40973a3-c862-4426-9ad4-301c6226c879 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:39.537838 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9e681-073c-49da-8200-a400a939a627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:39.538746 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9e681-073c-49da-8200-a400a939a627 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:39.539207 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9e681-073c-49da-8200-a400a939a627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:39.540269 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9e681-073c-49da-8200-a400a939a627 admin admin] PATH_INFO: `/v3/projects/059c5427c4a74081bed65dbc46045c2b/users/bdc198931ec14e81861ad78d45589a96/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:39.558903 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc9e681-073c-49da-8200-a400a939a627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=059c5427c4a74081bed65dbc46045c2b, user_id=bdc198931ec14e81861ad78d45589a96, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:39.561764 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc9e681-073c-49da-8200-a400a939a627 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:39.577346 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1385/2770] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:41:39 2022] PUT /identity/v3/projects/059c5427c4a74081bed65dbc46045c2b/users/bdc198931ec14e81861ad78d45589a96/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:39.588404 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3d83a-10d7-46d8-97fa-a97a4aa3c58e None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:39.588726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3d83a-10d7-46d8-97fa-a97a4aa3c58e None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:39.588956 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3d83a-10d7-46d8-97fa-a97a4aa3c58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:39.620471 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-99e3d83a-10d7-46d8-97fa-a97a4aa3c58e None None] MFA Rules not processed for user `bdc198931ec14e81861ad78d45589a96`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:39.621738 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-99e3d83a-10d7-46d8-97fa-a97a4aa3c58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:39.698875 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1386/2771] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:41:39.703432 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9e681-073c-49da-8200-a400a939a627 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:39.779826 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9e681-073c-49da-8200-a400a939a627 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:39.784522 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15820a5a-c32c-4ce5-b51c-bd3b9821d223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:39.785818 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15820a5a-c32c-4ce5-b51c-bd3b9821d223 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:39.786104 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15820a5a-c32c-4ce5-b51c-bd3b9821d223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:39.786408 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15820a5a-c32c-4ce5-b51c-bd3b9821d223 admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:39.787126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15820a5a-c32c-4ce5-b51c-bd3b9821d223 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:39.789145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15820a5a-c32c-4ce5-b51c-bd3b9821d223 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:39.842117 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1386/2772] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:41:39 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:39.846978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3d83a-10d7-46d8-97fa-a97a4aa3c58e None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:39.867780 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3d83a-10d7-46d8-97fa-a97a4aa3c58e None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:39.953731 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c06d4-8af2-4b26-8b3b-897851d99fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:39.955576 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c06d4-8af2-4b26-8b3b-897851d99fed admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:39.956065 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c06d4-8af2-4b26-8b3b-897851d99fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:39.956538 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c06d4-8af2-4b26-8b3b-897851d99fed admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:39.957383 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c06d4-8af2-4b26-8b3b-897851d99fed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:39.966653 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c06d4-8af2-4b26-8b3b-897851d99fed admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:40.002618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1387/2773] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:41:39 2022] POST /identity/v3/users => generated 288 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:40.006806 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15820a5a-c32c-4ce5-b51c-bd3b9821d223 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:40.018731 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15820a5a-c32c-4ce5-b51c-bd3b9821d223 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:40.025546 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1affa8b-69ec-4b6e-a9d8-24d696872b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:40.026190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1affa8b-69ec-4b6e-a9d8-24d696872b61 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:40.026588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1affa8b-69ec-4b6e-a9d8-24d696872b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:40.026868 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1affa8b-69ec-4b6e-a9d8-24d696872b61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:40.027688 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1affa8b-69ec-4b6e-a9d8-24d696872b61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:40.029444 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1affa8b-69ec-4b6e-a9d8-24d696872b61 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:40.042806 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1387/2774] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:40.047204 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c06d4-8af2-4b26-8b3b-897851d99fed admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:40.061163 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c06d4-8af2-4b26-8b3b-897851d99fed admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:40.068005 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f6e1cf-aaa5-49d5-86ce-af6879aa246d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:40.068847 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6e1cf-aaa5-49d5-86ce-af6879aa246d admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:40.069171 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6e1cf-aaa5-49d5-86ce-af6879aa246d admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:40.069416 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6e1cf-aaa5-49d5-86ce-af6879aa246d admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:40.070107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f6e1cf-aaa5-49d5-86ce-af6879aa246d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:40.072565 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f6e1cf-aaa5-49d5-86ce-af6879aa246d admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:40.086267 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1388/2775] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:40.090967 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1affa8b-69ec-4b6e-a9d8-24d696872b61 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:40.103487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1affa8b-69ec-4b6e-a9d8-24d696872b61 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:40.108566 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66765b3f-f335-4371-acb6-7bcf8072aa3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:40.109253 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66765b3f-f335-4371-acb6-7bcf8072aa3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:40.109538 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66765b3f-f335-4371-acb6-7bcf8072aa3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:40.109815 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66765b3f-f335-4371-acb6-7bcf8072aa3f admin admin] PATH_INFO: `/v3/projects/98943d12ab32478da85bac0984573be4/users/37754e7dcbcf452b9120d72acb9452b3/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:40.130924 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66765b3f-f335-4371-acb6-7bcf8072aa3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=98943d12ab32478da85bac0984573be4, user_id=37754e7dcbcf452b9120d72acb9452b3, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:40.134975 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66765b3f-f335-4371-acb6-7bcf8072aa3f admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:40.154052 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1388/2776] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:41:40 2022] PUT /identity/v3/projects/98943d12ab32478da85bac0984573be4/users/37754e7dcbcf452b9120d72acb9452b3/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:40.164628 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118e4a1-73d7-45ea-afcf-90c6b0b2b98c None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:40.165019 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118e4a1-73d7-45ea-afcf-90c6b0b2b98c None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:40.166008 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118e4a1-73d7-45ea-afcf-90c6b0b2b98c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:40.206943 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-f118e4a1-73d7-45ea-afcf-90c6b0b2b98c None None] MFA Rules not processed for user `37754e7dcbcf452b9120d72acb9452b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:40.208354 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f118e4a1-73d7-45ea-afcf-90c6b0b2b98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:40.216184 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:40.313156 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1389/2777] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:40.320307 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66765b3f-f335-4371-acb6-7bcf8072aa3f admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:40.386118 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:40.428735 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66765b3f-f335-4371-acb6-7bcf8072aa3f admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:40.438106 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27339ca9-5996-405d-afd7-84d264641d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:40.440211 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27339ca9-5996-405d-afd7-84d264641d9a admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:40.441171 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27339ca9-5996-405d-afd7-84d264641d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:40.441171 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27339ca9-5996-405d-afd7-84d264641d9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:40.442264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27339ca9-5996-405d-afd7-84d264641d9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:40.444668 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27339ca9-5996-405d-afd7-84d264641d9a admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:40.486433 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1389/2778] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:41:40 2022] POST /identity/v3/projects => generated 377 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:40.490199 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118e4a1-73d7-45ea-afcf-90c6b0b2b98c None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:40.508984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118e4a1-73d7-45ea-afcf-90c6b0b2b98c None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:40.583902 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac65818-4e5d-4e8a-bf58-120688bebd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:40.584548 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac65818-4e5d-4e8a-bf58-120688bebd01 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:40.585369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac65818-4e5d-4e8a-bf58-120688bebd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:40.585369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac65818-4e5d-4e8a-bf58-120688bebd01 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:40.585903 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac65818-4e5d-4e8a-bf58-120688bebd01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:40.588503 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac65818-4e5d-4e8a-bf58-120688bebd01 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:40.615204 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1390/2779] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:41:40 2022] POST /identity/v3/users => generated 295 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:40.619524 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27339ca9-5996-405d-afd7-84d264641d9a admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:40.632311 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27339ca9-5996-405d-afd7-84d264641d9a admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:40.638755 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb6409-1b62-4d9b-aeb0-ee7be537a2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:40.639398 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb6409-1b62-4d9b-aeb0-ee7be537a2e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:40.639681 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb6409-1b62-4d9b-aeb0-ee7be537a2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:40.639959 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb6409-1b62-4d9b-aeb0-ee7be537a2e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:40.640660 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb6409-1b62-4d9b-aeb0-ee7be537a2e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:40.642780 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb6409-1b62-4d9b-aeb0-ee7be537a2e8 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:40.655619 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1390/2780] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:40.660080 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac65818-4e5d-4e8a-bf58-120688bebd01 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:40.673047 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac65818-4e5d-4e8a-bf58-120688bebd01 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:40.678555 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c9ec00-445e-4a93-b5a0-24ecb3e5daab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:40.679563 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9ec00-445e-4a93-b5a0-24ecb3e5daab admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:40.679790 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9ec00-445e-4a93-b5a0-24ecb3e5daab admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:40.680068 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9ec00-445e-4a93-b5a0-24ecb3e5daab admin admin] PATH_INFO: `/v3/projects/e845013156cb4d3aa04656150323b378/users/a0b57f40900c4a0994c8d5351bb37076/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:40.699764 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c9ec00-445e-4a93-b5a0-24ecb3e5daab admin admin] RBAC: Authorizing `identity:create_grant(project_id=e845013156cb4d3aa04656150323b378, user_id=a0b57f40900c4a0994c8d5351bb37076, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:40.703788 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c9ec00-445e-4a93-b5a0-24ecb3e5daab admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:40.720021 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1391/2781] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:41:40 2022] PUT /identity/v3/projects/e845013156cb4d3aa04656150323b378/users/a0b57f40900c4a0994c8d5351bb37076/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:40.724057 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb6409-1b62-4d9b-aeb0-ee7be537a2e8 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:40.734988 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb6409-1b62-4d9b-aeb0-ee7be537a2e8 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:40.817779 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db41ac31-943e-4445-b20f-31165b5880b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:40.818649 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41ac31-943e-4445-b20f-31165b5880b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:40.819026 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41ac31-943e-4445-b20f-31165b5880b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:40.819425 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41ac31-943e-4445-b20f-31165b5880b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:40.821467 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db41ac31-943e-4445-b20f-31165b5880b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:40.822681 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db41ac31-943e-4445-b20f-31165b5880b8 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:40.834306 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1391/2782] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:40.838165 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c9ec00-445e-4a93-b5a0-24ecb3e5daab admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:40.848564 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c9ec00-445e-4a93-b5a0-24ecb3e5daab admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:40.867355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dcb4a6-a7db-431a-8ad5-41a57cc9cb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:40.868763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dcb4a6-a7db-431a-8ad5-41a57cc9cb01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:40.869658 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dcb4a6-a7db-431a-8ad5-41a57cc9cb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:40.869951 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dcb4a6-a7db-431a-8ad5-41a57cc9cb01 admin admin] PATH_INFO: `/v3/projects/e845013156cb4d3aa04656150323b378/users/a0b57f40900c4a0994c8d5351bb37076/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:40.873219 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dcb4a6-a7db-431a-8ad5-41a57cc9cb01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e845013156cb4d3aa04656150323b378, user_id=a0b57f40900c4a0994c8d5351bb37076, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:40.876311 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dcb4a6-a7db-431a-8ad5-41a57cc9cb01 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:40.893996 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1392/2783] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:41:40 2022] PUT /identity/v3/projects/e845013156cb4d3aa04656150323b378/users/a0b57f40900c4a0994c8d5351bb37076/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:41:40.964624 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6e734-a49f-4fb0-9088-5afe3e2f98ba None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:40.964914 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6e734-a49f-4fb0-9088-5afe3e2f98ba None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:40.965209 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6e734-a49f-4fb0-9088-5afe3e2f98ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:41.003074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-78f6e734-a49f-4fb0-9088-5afe3e2f98ba None None] MFA Rules not processed for user `a0b57f40900c4a0994c8d5351bb37076`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:41.004497 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-78f6e734-a49f-4fb0-9088-5afe3e2f98ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:41.109250 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1392/2784] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:40 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:41.120616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dcb4a6-a7db-431a-8ad5-41a57cc9cb01 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:41.122641 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-56dcb4a6-a7db-431a-8ad5-41a57cc9cb01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:41.207801 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dcb4a6-a7db-431a-8ad5-41a57cc9cb01 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:41.213084 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d439c20-8c8c-41e0-b330-51d4b239b62a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:41.214063 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d439c20-8c8c-41e0-b330-51d4b239b62a service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:41.214524 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d439c20-8c8c-41e0-b330-51d4b239b62a service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:41.214899 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d439c20-8c8c-41e0-b330-51d4b239b62a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:41.229263 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:41.310462 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d439c20-8c8c-41e0-b330-51d4b239b62a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:41.311954 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d439c20-8c8c-41e0-b330-51d4b239b62a service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:41.322091 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1393/2785] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:41.466000 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09cbe12-fd64-48ce-b8a1-00de05d509b4 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:41.466554 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09cbe12-fd64-48ce-b8a1-00de05d509b4 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:41.467326 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09cbe12-fd64-48ce-b8a1-00de05d509b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:41.503187 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-f09cbe12-fd64-48ce-b8a1-00de05d509b4 None None] MFA Rules not processed for user `a237b400429f4200909581e3c5e0cbfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:41.504422 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-f09cbe12-fd64-48ce-b8a1-00de05d509b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:41.527451 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:41.593099 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1393/2786] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:41.614092 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d439c20-8c8c-41e0-b330-51d4b239b62a service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:41.615930 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0d439c20-8c8c-41e0-b330-51d4b239b62a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:41.733771 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d439c20-8c8c-41e0-b330-51d4b239b62a service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:41.742501 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00309f64-b9f4-43de-946b-ad3f5b354701 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:41.743654 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00309f64-b9f4-43de-946b-ad3f5b354701 service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:41.743960 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00309f64-b9f4-43de-946b-ad3f5b354701 service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:41.744298 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00309f64-b9f4-43de-946b-ad3f5b354701 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:41.754131 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00309f64-b9f4-43de-946b-ad3f5b354701 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:41.755857 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00309f64-b9f4-43de-946b-ad3f5b354701 service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:41.766562 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1394/2787] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:42.953176 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09cbe12-fd64-48ce-b8a1-00de05d509b4 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:42.955702 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-f09cbe12-fd64-48ce-b8a1-00de05d509b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:42.972334 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09cbe12-fd64-48ce-b8a1-00de05d509b4 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:42.977388 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ffd7d-4e84-4c7f-9b8d-6f080ae25346 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:42.978000 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ffd7d-4e84-4c7f-9b8d-6f080ae25346 service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:42.978340 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ffd7d-4e84-4c7f-9b8d-6f080ae25346 service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:42.978599 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ffd7d-4e84-4c7f-9b8d-6f080ae25346 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:43.074839 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ffd7d-4e84-4c7f-9b8d-6f080ae25346 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:43.076668 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ffd7d-4e84-4c7f-9b8d-6f080ae25346 service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:43.089890 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1394/2788] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:43.172988 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e02cc-dae1-428e-aee8-754b61eec827 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:43.173510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e02cc-dae1-428e-aee8-754b61eec827 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:43.173639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e02cc-dae1-428e-aee8-754b61eec827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:43.195749 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-444e02cc-dae1-428e-aee8-754b61eec827 None None] MFA Rules not processed for user `a237b400429f4200909581e3c5e0cbfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:43.197003 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-444e02cc-dae1-428e-aee8-754b61eec827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:43.208969 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1395/2789] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:43.748644 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ffd7d-4e84-4c7f-9b8d-6f080ae25346 service cinder] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:43.757759 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ffd7d-4e84-4c7f-9b8d-6f080ae25346 service cinder] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:43.762215 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ceef50-ced5-48b9-bbe1-9f8f7b365f56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:43.762917 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ceef50-ced5-48b9-bbe1-9f8f7b365f56 service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:43.763224 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ceef50-ced5-48b9-bbe1-9f8f7b365f56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:43.763532 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ceef50-ced5-48b9-bbe1-9f8f7b365f56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:43.776050 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ceef50-ced5-48b9-bbe1-9f8f7b365f56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:43.777888 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ceef50-ced5-48b9-bbe1-9f8f7b365f56 service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:43.790470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1395/2790] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:47.894008 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e02cc-dae1-428e-aee8-754b61eec827 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:47.964567 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e02cc-dae1-428e-aee8-754b61eec827 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:47.969012 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29f333-848f-4e9a-ba87-1048a363b4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:47.969646 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29f333-848f-4e9a-ba87-1048a363b4e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:47.969928 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29f333-848f-4e9a-ba87-1048a363b4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:47.970207 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29f333-848f-4e9a-ba87-1048a363b4e5 admin admin] PATH_INFO: `/v3/users/bdc198931ec14e81861ad78d45589a96` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:47.971355 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a29f333-848f-4e9a-ba87-1048a363b4e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdc198931ec14e81861ad78d45589a96)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:47.973265 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a29f333-848f-4e9a-ba87-1048a363b4e5 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:48.012435 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:48.022615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1a29f333-848f-4e9a-ba87-1048a363b4e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdc198931ec14e81861ad78d45589a96'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:48.030014 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1a29f333-848f-4e9a-ba87-1048a363b4e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdc198931ec14e81861ad78d45589a96'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:48.030647 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1a29f333-848f-4e9a-ba87-1048a363b4e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdc198931ec14e81861ad78d45589a96'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:48.043430 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1a29f333-848f-4e9a-ba87-1048a363b4e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdc198931ec14e81861ad78d45589a96'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:48.044145 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1a29f333-848f-4e9a-ba87-1048a363b4e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdc198931ec14e81861ad78d45589a96'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:48.057877 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1396/2791] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:41:47 2022] DELETE /identity/v3/users/bdc198931ec14e81861ad78d45589a96 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:48.062358 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1384eb1-6506-43c6-b242-c4d992b9bcb1 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:48.062853 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1384eb1-6506-43c6-b242-c4d992b9bcb1 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:48.062965 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1384eb1-6506-43c6-b242-c4d992b9bcb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:48.082034 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-e1384eb1-6506-43c6-b242-c4d992b9bcb1 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:48.083340 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-e1384eb1-6506-43c6-b242-c4d992b9bcb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:48.144211 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1396/2792] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:48.154539 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29f333-848f-4e9a-ba87-1048a363b4e5 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:48.155961 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-1a29f333-848f-4e9a-ba87-1048a363b4e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:48.211912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29f333-848f-4e9a-ba87-1048a363b4e5 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:48.215607 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4802f965-4a84-4ee7-89df-dba73504ed83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:48.216238 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802f965-4a84-4ee7-89df-dba73504ed83 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:48.216518 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802f965-4a84-4ee7-89df-dba73504ed83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:48.216791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802f965-4a84-4ee7-89df-dba73504ed83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:48.225634 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4802f965-4a84-4ee7-89df-dba73504ed83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:48.227826 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4802f965-4a84-4ee7-89df-dba73504ed83 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:48.237009 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1397/2793] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:48.609491 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1384eb1-6506-43c6-b242-c4d992b9bcb1 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:48.611072 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-e1384eb1-6506-43c6-b242-c4d992b9bcb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:48.624173 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1384eb1-6506-43c6-b242-c4d992b9bcb1 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:48.630231 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce98bfd-2cb7-4c30-9366-957c8b7fcd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:48.631451 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce98bfd-2cb7-4c30-9366-957c8b7fcd55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:48.632148 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce98bfd-2cb7-4c30-9366-957c8b7fcd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:48.632253 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce98bfd-2cb7-4c30-9366-957c8b7fcd55 admin admin] PATH_INFO: `/v3/projects/059c5427c4a74081bed65dbc46045c2b` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:48.634301 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce98bfd-2cb7-4c30-9366-957c8b7fcd55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=059c5427c4a74081bed65dbc46045c2b)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:48.635976 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce98bfd-2cb7-4c30-9366-957c8b7fcd55 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:48.689065 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:48.718762 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-1ce98bfd-2cb7-4c30-9366-957c8b7fcd55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '059c5427c4a74081bed65dbc46045c2b', 'request_id': 'req-1ce98bfd-2cb7-4c30-9366-957c8b7fcd55'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:48.727246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-1ce98bfd-2cb7-4c30-9366-957c8b7fcd55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '059c5427c4a74081bed65dbc46045c2b', 'request_id': 'req-1ce98bfd-2cb7-4c30-9366-957c8b7fcd55'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:48.734142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-1ce98bfd-2cb7-4c30-9366-957c8b7fcd55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '059c5427c4a74081bed65dbc46045c2b', 'request_id': 'req-1ce98bfd-2cb7-4c30-9366-957c8b7fcd55'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:48.747053 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-1ce98bfd-2cb7-4c30-9366-957c8b7fcd55 admin admin] The token cache is being invalidate because project 059c5427c4a74081bed65dbc46045c2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:41:48.747443 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-1ce98bfd-2cb7-4c30-9366-957c8b7fcd55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:48.748171 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-1ce98bfd-2cb7-4c30-9366-957c8b7fcd55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:48.749708 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1397/2794] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:41:48 2022] DELETE /identity/v3/projects/059c5427c4a74081bed65dbc46045c2b => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:48.753674 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4802f965-4a84-4ee7-89df-dba73504ed83 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:48.763740 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4802f965-4a84-4ee7-89df-dba73504ed83 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:48.822676 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4a1c0-ec65-43c4-bef0-3cd7f7b2c25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:48.823317 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4a1c0-ec65-43c4-bef0-3cd7f7b2c25d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:48.823596 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4a1c0-ec65-43c4-bef0-3cd7f7b2c25d admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:48.823870 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4a1c0-ec65-43c4-bef0-3cd7f7b2c25d admin admin] PATH_INFO: `/v3/users/37754e7dcbcf452b9120d72acb9452b3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:48.824986 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba4a1c0-ec65-43c4-bef0-3cd7f7b2c25d admin admin] RBAC: Authorizing `identity:delete_user(user_id=37754e7dcbcf452b9120d72acb9452b3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:48.826990 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba4a1c0-ec65-43c4-bef0-3cd7f7b2c25d admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:48.886135 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1ba4a1c0-ec65-43c4-bef0-3cd7f7b2c25d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37754e7dcbcf452b9120d72acb9452b3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:48.893894 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1ba4a1c0-ec65-43c4-bef0-3cd7f7b2c25d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37754e7dcbcf452b9120d72acb9452b3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:48.894544 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1ba4a1c0-ec65-43c4-bef0-3cd7f7b2c25d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37754e7dcbcf452b9120d72acb9452b3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:48.896728 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:48.910761 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1ba4a1c0-ec65-43c4-bef0-3cd7f7b2c25d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37754e7dcbcf452b9120d72acb9452b3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:48.911803 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-1ba4a1c0-ec65-43c4-bef0-3cd7f7b2c25d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37754e7dcbcf452b9120d72acb9452b3'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:48.924303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1398/2795] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:41:48 2022] DELETE /identity/v3/users/37754e7dcbcf452b9120d72acb9452b3 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:41:49.224419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce98bfd-2cb7-4c30-9366-957c8b7fcd55 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:49.226256 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-1ce98bfd-2cb7-4c30-9366-957c8b7fcd55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:49.239754 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce98bfd-2cb7-4c30-9366-957c8b7fcd55 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:49.307362 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f8291-d9b3-48ed-a1b8-736051a65d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:49.308051 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f8291-d9b3-48ed-a1b8-736051a65d64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:49.308331 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f8291-d9b3-48ed-a1b8-736051a65d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:49.308604 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f8291-d9b3-48ed-a1b8-736051a65d64 admin admin] PATH_INFO: `/v3/projects/98943d12ab32478da85bac0984573be4` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:49.310106 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f8291-d9b3-48ed-a1b8-736051a65d64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98943d12ab32478da85bac0984573be4)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:49.312211 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f8291-d9b3-48ed-a1b8-736051a65d64 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:49.363875 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-490f8291-d9b3-48ed-a1b8-736051a65d64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98943d12ab32478da85bac0984573be4', 'request_id': 'req-490f8291-d9b3-48ed-a1b8-736051a65d64'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:49.372776 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-490f8291-d9b3-48ed-a1b8-736051a65d64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98943d12ab32478da85bac0984573be4', 'request_id': 'req-490f8291-d9b3-48ed-a1b8-736051a65d64'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:49.379490 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-490f8291-d9b3-48ed-a1b8-736051a65d64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98943d12ab32478da85bac0984573be4', 'request_id': 'req-490f8291-d9b3-48ed-a1b8-736051a65d64'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:49.390469 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-490f8291-d9b3-48ed-a1b8-736051a65d64 admin admin] The token cache is being invalidate because project 98943d12ab32478da85bac0984573be4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:41:49.390701 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-490f8291-d9b3-48ed-a1b8-736051a65d64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:49.391341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-490f8291-d9b3-48ed-a1b8-736051a65d64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:49.392819 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1398/2796] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:41:49 2022] DELETE /identity/v3/projects/98943d12ab32478da85bac0984573be4 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:49.396781 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4a1c0-ec65-43c4-bef0-3cd7f7b2c25d admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:49.398199 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-1ba4a1c0-ec65-43c4-bef0-3cd7f7b2c25d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:49.411081 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4a1c0-ec65-43c4-bef0-3cd7f7b2c25d admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:49.466714 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3facd265-d773-4b70-8437-2b547445d99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:49.467392 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3facd265-d773-4b70-8437-2b547445d99f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:49.467679 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3facd265-d773-4b70-8437-2b547445d99f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:49.467955 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3facd265-d773-4b70-8437-2b547445d99f admin admin] PATH_INFO: `/v3/users/a0b57f40900c4a0994c8d5351bb37076` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:49.469167 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3facd265-d773-4b70-8437-2b547445d99f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0b57f40900c4a0994c8d5351bb37076)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:49.471182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3facd265-d773-4b70-8437-2b547445d99f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:49.518206 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3facd265-d773-4b70-8437-2b547445d99f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0b57f40900c4a0994c8d5351bb37076'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:49.525898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3facd265-d773-4b70-8437-2b547445d99f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0b57f40900c4a0994c8d5351bb37076'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:49.527345 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3facd265-d773-4b70-8437-2b547445d99f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0b57f40900c4a0994c8d5351bb37076'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:49.540879 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3facd265-d773-4b70-8437-2b547445d99f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0b57f40900c4a0994c8d5351bb37076'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:49.541606 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-3facd265-d773-4b70-8437-2b547445d99f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0b57f40900c4a0994c8d5351bb37076'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:49.554056 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1399/2797] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:41:49 2022] DELETE /identity/v3/users/a0b57f40900c4a0994c8d5351bb37076 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:49.850992 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f8291-d9b3-48ed-a1b8-736051a65d64 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:49.853648 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-490f8291-d9b3-48ed-a1b8-736051a65d64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:49.875410 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f8291-d9b3-48ed-a1b8-736051a65d64 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:49.931203 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:49.938339 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d54696-1d66-4313-ba74-abe7814bdcc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:49.939107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d54696-1d66-4313-ba74-abe7814bdcc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:49.939608 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d54696-1d66-4313-ba74-abe7814bdcc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:49.939994 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d54696-1d66-4313-ba74-abe7814bdcc5 admin admin] PATH_INFO: `/v3/projects/e845013156cb4d3aa04656150323b378` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:49.941166 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d54696-1d66-4313-ba74-abe7814bdcc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e845013156cb4d3aa04656150323b378)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:49.943134 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d54696-1d66-4313-ba74-abe7814bdcc5 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:49.993427 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-16d54696-1d66-4313-ba74-abe7814bdcc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e845013156cb4d3aa04656150323b378', 'request_id': 'req-16d54696-1d66-4313-ba74-abe7814bdcc5'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:50.001793 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-16d54696-1d66-4313-ba74-abe7814bdcc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e845013156cb4d3aa04656150323b378', 'request_id': 'req-16d54696-1d66-4313-ba74-abe7814bdcc5'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:50.008235 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-16d54696-1d66-4313-ba74-abe7814bdcc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e845013156cb4d3aa04656150323b378', 'request_id': 'req-16d54696-1d66-4313-ba74-abe7814bdcc5'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:50.018676 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-16d54696-1d66-4313-ba74-abe7814bdcc5 admin admin] The token cache is being invalidate because project e845013156cb4d3aa04656150323b378 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:41:50.018967 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-16d54696-1d66-4313-ba74-abe7814bdcc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:50.019610 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-16d54696-1d66-4313-ba74-abe7814bdcc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:50.021099 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1399/2798] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:41:49 2022] DELETE /identity/v3/projects/e845013156cb4d3aa04656150323b378 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:50.026595 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44411c-ca5a-4623-9ce1-c5fdd37b5e0a None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:50.027067 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44411c-ca5a-4623-9ce1-c5fdd37b5e0a None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:50.027293 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44411c-ca5a-4623-9ce1-c5fdd37b5e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:50.047869 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-cc44411c-ca5a-4623-9ce1-c5fdd37b5e0a None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:50.049099 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-cc44411c-ca5a-4623-9ce1-c5fdd37b5e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:50.096838 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:50.111453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1400/2799] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:50.119464 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bda515-391f-4e18-bf59-c00d499e1f02 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:50.119758 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bda515-391f-4e18-bf59-c00d499e1f02 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:50.120038 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bda515-391f-4e18-bf59-c00d499e1f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:50.140035 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-26bda515-391f-4e18-bf59-c00d499e1f02 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:50.141267 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-26bda515-391f-4e18-bf59-c00d499e1f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:50.150225 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1400/2800] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:50.154493 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44411c-ca5a-4623-9ce1-c5fdd37b5e0a None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:50.164114 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44411c-ca5a-4623-9ce1-c5fdd37b5e0a None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:50.168620 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b6927-c159-4de8-86d2-c0fb9b6acfc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:50.169226 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b6927-c159-4de8-86d2-c0fb9b6acfc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:50.169510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b6927-c159-4de8-86d2-c0fb9b6acfc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:50.169787 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b6927-c159-4de8-86d2-c0fb9b6acfc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:50.170501 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b6927-c159-4de8-86d2-c0fb9b6acfc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:41:50.170812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b6927-c159-4de8-86d2-c0fb9b6acfc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:50.172408 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b6927-c159-4de8-86d2-c0fb9b6acfc8 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:50.184079 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1401/2801] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:41:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:41:50.188301 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bda515-391f-4e18-bf59-c00d499e1f02 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:50.198192 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bda515-391f-4e18-bf59-c00d499e1f02 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:50.203006 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1683ccc-f05f-40a1-a5bd-1911992f9c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:50.203601 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1683ccc-f05f-40a1-a5bd-1911992f9c8e admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:50.203882 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1683ccc-f05f-40a1-a5bd-1911992f9c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:50.204153 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1683ccc-f05f-40a1-a5bd-1911992f9c8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:50.204894 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1683ccc-f05f-40a1-a5bd-1911992f9c8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:50.206841 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1683ccc-f05f-40a1-a5bd-1911992f9c8e admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:50.235587 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1401/2802] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:41:50 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:50.239775 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b6927-c159-4de8-86d2-c0fb9b6acfc8 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:50.249902 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b6927-c159-4de8-86d2-c0fb9b6acfc8 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:50.306469 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3547c60-215b-46ce-a079-fc68001cf737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:50.307090 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3547c60-215b-46ce-a079-fc68001cf737 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:50.307376 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3547c60-215b-46ce-a079-fc68001cf737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:50.307650 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3547c60-215b-46ce-a079-fc68001cf737 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:50.308289 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3547c60-215b-46ce-a079-fc68001cf737 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:50.310206 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3547c60-215b-46ce-a079-fc68001cf737 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:50.332047 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1402/2803] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:41:50 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:50.336954 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1683ccc-f05f-40a1-a5bd-1911992f9c8e admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:50.347308 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1683ccc-f05f-40a1-a5bd-1911992f9c8e admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:50.352534 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769f184-72bb-496c-9968-b28c8a9b7c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:50.353145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769f184-72bb-496c-9968-b28c8a9b7c0a admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:50.353421 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769f184-72bb-496c-9968-b28c8a9b7c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:50.353740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769f184-72bb-496c-9968-b28c8a9b7c0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:50.354472 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769f184-72bb-496c-9968-b28c8a9b7c0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:50.356085 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769f184-72bb-496c-9968-b28c8a9b7c0a admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:50.367160 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1402/2804] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:50.371892 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3547c60-215b-46ce-a079-fc68001cf737 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:50.381429 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3547c60-215b-46ce-a079-fc68001cf737 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:50.386589 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce1f36-6466-4d7a-976f-ec7786c31835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:50.387199 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce1f36-6466-4d7a-976f-ec7786c31835 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:50.387477 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce1f36-6466-4d7a-976f-ec7786c31835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:50.387751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce1f36-6466-4d7a-976f-ec7786c31835 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:50.388391 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce1f36-6466-4d7a-976f-ec7786c31835 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:50.389975 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce1f36-6466-4d7a-976f-ec7786c31835 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:50.401448 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1403/2805] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:50.405690 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769f184-72bb-496c-9968-b28c8a9b7c0a admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:50.415830 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769f184-72bb-496c-9968-b28c8a9b7c0a admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:50.420552 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276102b1-afaf-4d48-9c96-dc3e6ce31f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:50.421206 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276102b1-afaf-4d48-9c96-dc3e6ce31f10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:50.421492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276102b1-afaf-4d48-9c96-dc3e6ce31f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:50.421771 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276102b1-afaf-4d48-9c96-dc3e6ce31f10 admin admin] PATH_INFO: `/v3/projects/ef3fae8a59d54d3e881716e8ff6a1764/users/424e5b3299ee4cf7b63126b1d2af5c92/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:50.436572 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276102b1-afaf-4d48-9c96-dc3e6ce31f10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef3fae8a59d54d3e881716e8ff6a1764, user_id=424e5b3299ee4cf7b63126b1d2af5c92, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:50.439240 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276102b1-afaf-4d48-9c96-dc3e6ce31f10 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:50.453183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1403/2806] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:41:50 2022] PUT /identity/v3/projects/ef3fae8a59d54d3e881716e8ff6a1764/users/424e5b3299ee4cf7b63126b1d2af5c92/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:50.465598 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff48da-57c6-4294-a1ce-b8ed41efa593 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:50.465887 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff48da-57c6-4294-a1ce-b8ed41efa593 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:50.466162 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff48da-57c6-4294-a1ce-b8ed41efa593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:50.494536 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-9aff48da-57c6-4294-a1ce-b8ed41efa593 None None] MFA Rules not processed for user `424e5b3299ee4cf7b63126b1d2af5c92`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:50.495786 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-9aff48da-57c6-4294-a1ce-b8ed41efa593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:50.560613 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1404/2807] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:41:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:50.571975 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276102b1-afaf-4d48-9c96-dc3e6ce31f10 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:50.573650 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-276102b1-afaf-4d48-9c96-dc3e6ce31f10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:50.631589 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276102b1-afaf-4d48-9c96-dc3e6ce31f10 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:50.635880 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a9cc8-ef5c-4988-af55-64e5be10e335 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:50.636485 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a9cc8-ef5c-4988-af55-64e5be10e335 service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:50.636773 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a9cc8-ef5c-4988-af55-64e5be10e335 service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:50.637045 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a9cc8-ef5c-4988-af55-64e5be10e335 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:50.645234 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a9cc8-ef5c-4988-af55-64e5be10e335 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:50.646714 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a9cc8-ef5c-4988-af55-64e5be10e335 service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:50.655465 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1404/2808] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:42:00.557773 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=75,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:00.651595 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=78,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:00.786813 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff48da-57c6-4294-a1ce-b8ed41efa593 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:00.889883 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff48da-57c6-4294-a1ce-b8ed41efa593 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:00.894534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e122d8-0c65-461d-a304-f5e942dce75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:00.895162 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e122d8-0c65-461d-a304-f5e942dce75a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:00.895442 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e122d8-0c65-461d-a304-f5e942dce75a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:00.895718 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e122d8-0c65-461d-a304-f5e942dce75a admin admin] PATH_INFO: `/v3/users/424e5b3299ee4cf7b63126b1d2af5c92` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:00.896846 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e122d8-0c65-461d-a304-f5e942dce75a admin admin] RBAC: Authorizing `identity:delete_user(user_id=424e5b3299ee4cf7b63126b1d2af5c92)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:00.899350 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e122d8-0c65-461d-a304-f5e942dce75a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:00.946530 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e1e122d8-0c65-461d-a304-f5e942dce75a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '424e5b3299ee4cf7b63126b1d2af5c92'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:00.954535 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e1e122d8-0c65-461d-a304-f5e942dce75a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '424e5b3299ee4cf7b63126b1d2af5c92'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:00.955161 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e1e122d8-0c65-461d-a304-f5e942dce75a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '424e5b3299ee4cf7b63126b1d2af5c92'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:00.968161 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e1e122d8-0c65-461d-a304-f5e942dce75a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '424e5b3299ee4cf7b63126b1d2af5c92'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:00.968838 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-e1e122d8-0c65-461d-a304-f5e942dce75a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '424e5b3299ee4cf7b63126b1d2af5c92'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:00.981042 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1405/2809] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:42:00 2022] DELETE /identity/v3/users/424e5b3299ee4cf7b63126b1d2af5c92 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:00.985594 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c3f37-b26b-4325-94d2-81389b025449 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:00.985877 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c3f37-b26b-4325-94d2-81389b025449 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:00.986180 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c3f37-b26b-4325-94d2-81389b025449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:01.038930 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-431c3f37-b26b-4325-94d2-81389b025449 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:01.040136 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-431c3f37-b26b-4325-94d2-81389b025449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:01.102257 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1405/2810] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:42:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:01.112001 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e122d8-0c65-461d-a304-f5e942dce75a admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:01.113471 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-e1e122d8-0c65-461d-a304-f5e942dce75a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:01.155844 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:01.189967 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e122d8-0c65-461d-a304-f5e942dce75a admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:01.193711 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251798ae-de62-4fc2-9aa0-c506283f7401 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:01.194282 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251798ae-de62-4fc2-9aa0-c506283f7401 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:01.194576 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251798ae-de62-4fc2-9aa0-c506283f7401 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:01.194875 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251798ae-de62-4fc2-9aa0-c506283f7401 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:01.204364 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251798ae-de62-4fc2-9aa0-c506283f7401 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:01.205821 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251798ae-de62-4fc2-9aa0-c506283f7401 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:01.215391 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1406/2811] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:01.481093 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c3f37-b26b-4325-94d2-81389b025449 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:01.482781 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-431c3f37-b26b-4325-94d2-81389b025449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:01.495917 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c3f37-b26b-4325-94d2-81389b025449 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:01.501344 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b23c8-ed4c-4008-83ad-a5e0f3c3943e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:01.502042 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b23c8-ed4c-4008-83ad-a5e0f3c3943e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:01.502409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b23c8-ed4c-4008-83ad-a5e0f3c3943e admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:01.503592 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b23c8-ed4c-4008-83ad-a5e0f3c3943e admin admin] PATH_INFO: `/v3/projects/ef3fae8a59d54d3e881716e8ff6a1764` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:01.504780 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2b23c8-ed4c-4008-83ad-a5e0f3c3943e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef3fae8a59d54d3e881716e8ff6a1764)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:01.507511 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2b23c8-ed4c-4008-83ad-a5e0f3c3943e admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:01.559513 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5e2b23c8-ed4c-4008-83ad-a5e0f3c3943e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef3fae8a59d54d3e881716e8ff6a1764', 'request_id': 'req-5e2b23c8-ed4c-4008-83ad-a5e0f3c3943e'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:01.569043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5e2b23c8-ed4c-4008-83ad-a5e0f3c3943e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef3fae8a59d54d3e881716e8ff6a1764', 'request_id': 'req-5e2b23c8-ed4c-4008-83ad-a5e0f3c3943e'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:01.575902 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5e2b23c8-ed4c-4008-83ad-a5e0f3c3943e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef3fae8a59d54d3e881716e8ff6a1764', 'request_id': 'req-5e2b23c8-ed4c-4008-83ad-a5e0f3c3943e'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:01.586446 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5e2b23c8-ed4c-4008-83ad-a5e0f3c3943e admin admin] The token cache is being invalidate because project ef3fae8a59d54d3e881716e8ff6a1764 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:42:01.586740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5e2b23c8-ed4c-4008-83ad-a5e0f3c3943e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:01.587384 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-5e2b23c8-ed4c-4008-83ad-a5e0f3c3943e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:01.588865 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1406/2812] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:42:01 2022] DELETE /identity/v3/projects/ef3fae8a59d54d3e881716e8ff6a1764 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:02.001395 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251798ae-de62-4fc2-9aa0-c506283f7401 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:02.002807 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-251798ae-de62-4fc2-9aa0-c506283f7401 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:02.060045 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251798ae-de62-4fc2-9aa0-c506283f7401 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:02.115948 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a87f3-9658-45b6-84f0-7ef4a187a723 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:02.116561 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a87f3-9658-45b6-84f0-7ef4a187a723 service cinder] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:02.116873 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a87f3-9658-45b6-84f0-7ef4a187a723 service cinder] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:02.117148 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a87f3-9658-45b6-84f0-7ef4a187a723 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:02.118901 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ce4a87f3-9658-45b6-84f0-7ef4a187a723 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:02.181730 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a87f3-9658-45b6-84f0-7ef4a187a723 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:02.183224 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a87f3-9658-45b6-84f0-7ef4a187a723 service cinder] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:02.192756 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1407/2813] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:09.065733 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d03430-a30b-45e2-b191-3f2eb4e24b7e None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:09.066220 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d03430-a30b-45e2-b191-3f2eb4e24b7e None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:09.066341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d03430-a30b-45e2-b191-3f2eb4e24b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:09.085951 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-63d03430-a30b-45e2-b191-3f2eb4e24b7e None None] MFA Rules not processed for user `a3059f5115bb4bad837daf9f18752143`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:09.087185 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-63d03430-a30b-45e2-b191-3f2eb4e24b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:09.094176 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1407/2814] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:42:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:10.390506 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a87f3-9658-45b6-84f0-7ef4a187a723 service cinder] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:10.391909 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ce4a87f3-9658-45b6-84f0-7ef4a187a723 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:10.431882 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:10.482601 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a87f3-9658-45b6-84f0-7ef4a187a723 service cinder] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:10.487600 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3726951-0215-44ce-934b-b9bbed428f39 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ddfb151bbf844efae73ec1990e7eaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:10.488204 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3726951-0215-44ce-934b-b9bbed428f39 service placement] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:10.488483 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3726951-0215-44ce-934b-b9bbed428f39 service placement] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:10.488755 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3726951-0215-44ce-934b-b9bbed428f39 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:10.490474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f3726951-0215-44ce-934b-b9bbed428f39 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:10.502591 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3726951-0215-44ce-934b-b9bbed428f39 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:10.504159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3726951-0215-44ce-934b-b9bbed428f39 service placement] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:10.518363 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1408/2815] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:19.080827 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=69,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:20.509924 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=25 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:34.442484 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d03430-a30b-45e2-b191-3f2eb4e24b7e None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:34.444563 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-63d03430-a30b-45e2-b191-3f2eb4e24b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:34.538790 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d03430-a30b-45e2-b191-3f2eb4e24b7e None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:34.542599 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675667e-d0bd-48b7-a58f-ad6376dd001c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:34.543286 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675667e-d0bd-48b7-a58f-ad6376dd001c service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:34.543567 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675667e-d0bd-48b7-a58f-ad6376dd001c service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:34.543840 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675667e-d0bd-48b7-a58f-ad6376dd001c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:34.545631 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-3675667e-d0bd-48b7-a58f-ad6376dd001c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:34.556184 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675667e-d0bd-48b7-a58f-ad6376dd001c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:34.557647 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675667e-d0bd-48b7-a58f-ad6376dd001c service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:34.565981 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1408/2816] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:34.642877 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3726951-0215-44ce-934b-b9bbed428f39 service placement] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:34.652362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3726951-0215-44ce-934b-b9bbed428f39 service placement] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:34.655939 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31352a-84c6-487e-8506-0afe4a82d416 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:34.656541 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31352a-84c6-487e-8506-0afe4a82d416 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:34.656819 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31352a-84c6-487e-8506-0afe4a82d416 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:34.657090 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31352a-84c6-487e-8506-0afe4a82d416 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:34.658823 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-eb31352a-84c6-487e-8506-0afe4a82d416 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:34.749600 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb31352a-84c6-487e-8506-0afe4a82d416 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:34.751014 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb31352a-84c6-487e-8506-0afe4a82d416 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:34.771578 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1409/2817] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:44.562332 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=35 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:44.768941 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=46 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:48.931063 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b360e-dab6-499e-bb95-9d7fc9bcb898 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:48.931591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b360e-dab6-499e-bb95-9d7fc9bcb898 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:48.931591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b360e-dab6-499e-bb95-9d7fc9bcb898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:48.952174 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-241b360e-dab6-499e-bb95-9d7fc9bcb898 None None] MFA Rules not processed for user `a237b400429f4200909581e3c5e0cbfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:48.953402 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-241b360e-dab6-499e-bb95-9d7fc9bcb898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:49.009083 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1409/2818] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:42:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:59.001688 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=25 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:05.854986 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3273ed7-a15e-4ba3-a2b4-f536c0f545be None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:05.855463 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3273ed7-a15e-4ba3-a2b4-f536c0f545be None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:05.855564 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3273ed7-a15e-4ba3-a2b4-f536c0f545be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:05.884983 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-c3273ed7-a15e-4ba3-a2b4-f536c0f545be None None] MFA Rules not processed for user `a5b3e53fd1314e17ad28c42a44e8eff3`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:05.886182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c3273ed7-a15e-4ba3-a2b4-f536c0f545be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:05.942021 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1410/2819] 10.222.0.18 () {58 vars in 967 bytes} [Thu Sep 1 16:43:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:05.983119 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b360e-dab6-499e-bb95-9d7fc9bcb898 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:05.984545 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-241b360e-dab6-499e-bb95-9d7fc9bcb898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:06.054175 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b360e-dab6-499e-bb95-9d7fc9bcb898 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:06.059132 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5906031-49d7-41d4-872f-4c411ba89e99 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8137f6b2ced040bd9752239c7e987f03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:06.059765 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5906031-49d7-41d4-872f-4c411ba89e99 service swift] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:06.060045 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5906031-49d7-41d4-872f-4c411ba89e99 service swift] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:06.060317 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5906031-49d7-41d4-872f-4c411ba89e99 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:06.068280 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5906031-49d7-41d4-872f-4c411ba89e99 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:06.069862 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5906031-49d7-41d4-872f-4c411ba89e99 service swift] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:06.077783 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1410/2820] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Sep 1 16:43:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:43:09.309588 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3273ed7-a15e-4ba3-a2b4-f536c0f545be None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:09.311351 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-c3273ed7-a15e-4ba3-a2b4-f536c0f545be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:09.333923 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3273ed7-a15e-4ba3-a2b4-f536c0f545be None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:09.340912 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034acf06-302b-42a6-862d-a9c5b9d35b90 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ddfb151bbf844efae73ec1990e7eaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:09.341542 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034acf06-302b-42a6-862d-a9c5b9d35b90 service placement] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:09.341821 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034acf06-302b-42a6-862d-a9c5b9d35b90 service placement] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:09.342090 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034acf06-302b-42a6-862d-a9c5b9d35b90 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:09.343985 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-034acf06-302b-42a6-862d-a9c5b9d35b90 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:09.368022 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034acf06-302b-42a6-862d-a9c5b9d35b90 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:09.368022 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034acf06-302b-42a6-862d-a9c5b9d35b90 service placement] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:09.377602 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1411/2821] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:12.687388 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5906031-49d7-41d4-872f-4c411ba89e99 service swift] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:12.688891 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-f5906031-49d7-41d4-872f-4c411ba89e99 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:12.764008 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5906031-49d7-41d4-872f-4c411ba89e99 service swift] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:12.768632 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7366fdf-b86d-4f1a-b075-4c3c534bc073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:12.769281 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7366fdf-b86d-4f1a-b075-4c3c534bc073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:12.769564 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7366fdf-b86d-4f1a-b075-4c3c534bc073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:12.769839 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7366fdf-b86d-4f1a-b075-4c3c534bc073 admin admin] PATH_INFO: `/v3/users/003d7b8eee13419780bb5e055a01bb77` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:12.771063 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7366fdf-b86d-4f1a-b075-4c3c534bc073 admin admin] RBAC: Authorizing `identity:delete_user(user_id=003d7b8eee13419780bb5e055a01bb77)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:12.772993 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7366fdf-b86d-4f1a-b075-4c3c534bc073 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:12.821564 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-a7366fdf-b86d-4f1a-b075-4c3c534bc073 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '003d7b8eee13419780bb5e055a01bb77'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:12.822348 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-a7366fdf-b86d-4f1a-b075-4c3c534bc073 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '003d7b8eee13419780bb5e055a01bb77'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:12.829412 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-a7366fdf-b86d-4f1a-b075-4c3c534bc073 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '003d7b8eee13419780bb5e055a01bb77'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:12.830043 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-a7366fdf-b86d-4f1a-b075-4c3c534bc073 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '003d7b8eee13419780bb5e055a01bb77'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:12.841994 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-a7366fdf-b86d-4f1a-b075-4c3c534bc073 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '003d7b8eee13419780bb5e055a01bb77'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:12.854727 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1411/2822] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:12 2022] DELETE /identity/v3/users/003d7b8eee13419780bb5e055a01bb77 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:13.044177 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034acf06-302b-42a6-862d-a9c5b9d35b90 service placement] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:13.058453 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034acf06-302b-42a6-862d-a9c5b9d35b90 service placement] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:13.120815 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82125e4c-c320-4e91-af5b-d3d903600aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:13.121442 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82125e4c-c320-4e91-af5b-d3d903600aa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:13.121725 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82125e4c-c320-4e91-af5b-d3d903600aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:13.122111 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82125e4c-c320-4e91-af5b-d3d903600aa9 admin admin] PATH_INFO: `/v3/projects/13ab20a85384485e95e2c0dadcc8b43c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:13.123266 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82125e4c-c320-4e91-af5b-d3d903600aa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13ab20a85384485e95e2c0dadcc8b43c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:13.125154 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82125e4c-c320-4e91-af5b-d3d903600aa9 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:13.170438 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-82125e4c-c320-4e91-af5b-d3d903600aa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13ab20a85384485e95e2c0dadcc8b43c', 'request_id': 'req-82125e4c-c320-4e91-af5b-d3d903600aa9'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.177033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-82125e4c-c320-4e91-af5b-d3d903600aa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13ab20a85384485e95e2c0dadcc8b43c', 'request_id': 'req-82125e4c-c320-4e91-af5b-d3d903600aa9'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.184055 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-82125e4c-c320-4e91-af5b-d3d903600aa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13ab20a85384485e95e2c0dadcc8b43c', 'request_id': 'req-82125e4c-c320-4e91-af5b-d3d903600aa9'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.194444 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-82125e4c-c320-4e91-af5b-d3d903600aa9 admin admin] The token cache is being invalidate because project 13ab20a85384485e95e2c0dadcc8b43c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:13.194775 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-82125e4c-c320-4e91-af5b-d3d903600aa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.195720 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-82125e4c-c320-4e91-af5b-d3d903600aa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.197209 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1412/2823] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:13 2022] DELETE /identity/v3/projects/13ab20a85384485e95e2c0dadcc8b43c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:13.201661 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7366fdf-b86d-4f1a-b075-4c3c534bc073 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:13.203313 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-a7366fdf-b86d-4f1a-b075-4c3c534bc073 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:13.217401 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7366fdf-b86d-4f1a-b075-4c3c534bc073 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:13.222144 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:13.287622 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2e666-b918-4b45-83a2-6f3713b2343e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:13.288271 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2e666-b918-4b45-83a2-6f3713b2343e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:13.288557 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2e666-b918-4b45-83a2-6f3713b2343e admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:13.288853 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2e666-b918-4b45-83a2-6f3713b2343e admin admin] PATH_INFO: `/v3/users/f6dd4e06e49a429fbef5601af067ce20` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:13.290121 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c2e666-b918-4b45-83a2-6f3713b2343e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6dd4e06e49a429fbef5601af067ce20)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:13.292466 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c2e666-b918-4b45-83a2-6f3713b2343e admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:13.347576 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-81c2e666-b918-4b45-83a2-6f3713b2343e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6dd4e06e49a429fbef5601af067ce20'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.347576 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-81c2e666-b918-4b45-83a2-6f3713b2343e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6dd4e06e49a429fbef5601af067ce20'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.360468 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-81c2e666-b918-4b45-83a2-6f3713b2343e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6dd4e06e49a429fbef5601af067ce20'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.361159 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-81c2e666-b918-4b45-83a2-6f3713b2343e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6dd4e06e49a429fbef5601af067ce20'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.375168 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-81c2e666-b918-4b45-83a2-6f3713b2343e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6dd4e06e49a429fbef5601af067ce20'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.387246 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1412/2824] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:13 2022] DELETE /identity/v3/users/f6dd4e06e49a429fbef5601af067ce20 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:13.562540 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82125e4c-c320-4e91-af5b-d3d903600aa9 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:13.564033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-82125e4c-c320-4e91-af5b-d3d903600aa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:13.577102 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82125e4c-c320-4e91-af5b-d3d903600aa9 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:13.587729 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:13.647499 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f89eff-4a27-4045-bdc2-7fc34d02a2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:13.648049 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f89eff-4a27-4045-bdc2-7fc34d02a2f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:13.648331 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f89eff-4a27-4045-bdc2-7fc34d02a2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:13.648603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f89eff-4a27-4045-bdc2-7fc34d02a2f9 admin admin] PATH_INFO: `/v3/projects/4cf6935593374d03a401512b9f31df9b` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:13.649714 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f89eff-4a27-4045-bdc2-7fc34d02a2f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cf6935593374d03a401512b9f31df9b)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:13.651652 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f89eff-4a27-4045-bdc2-7fc34d02a2f9 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:13.698414 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-a1f89eff-4a27-4045-bdc2-7fc34d02a2f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cf6935593374d03a401512b9f31df9b', 'request_id': 'req-a1f89eff-4a27-4045-bdc2-7fc34d02a2f9'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.704571 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-a1f89eff-4a27-4045-bdc2-7fc34d02a2f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cf6935593374d03a401512b9f31df9b', 'request_id': 'req-a1f89eff-4a27-4045-bdc2-7fc34d02a2f9'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.713785 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-a1f89eff-4a27-4045-bdc2-7fc34d02a2f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cf6935593374d03a401512b9f31df9b', 'request_id': 'req-a1f89eff-4a27-4045-bdc2-7fc34d02a2f9'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.725914 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-a1f89eff-4a27-4045-bdc2-7fc34d02a2f9 admin admin] The token cache is being invalidate because project 4cf6935593374d03a401512b9f31df9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:13.726211 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-a1f89eff-4a27-4045-bdc2-7fc34d02a2f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.726904 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-a1f89eff-4a27-4045-bdc2-7fc34d02a2f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.728396 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1413/2825] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:13 2022] DELETE /identity/v3/projects/4cf6935593374d03a401512b9f31df9b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:23.383709 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:23.721507 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=45,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:49.104140 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de788425-3890-44f4-94e7-d35f4a19c395 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:49.104666 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de788425-3890-44f4-94e7-d35f4a19c395 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:49.104666 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de788425-3890-44f4-94e7-d35f4a19c395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:49.123686 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-de788425-3890-44f4-94e7-d35f4a19c395 None None] MFA Rules not processed for user `a237b400429f4200909581e3c5e0cbfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:49.124886 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-de788425-3890-44f4-94e7-d35f4a19c395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:49.181010 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1413/2826] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:43:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:55.560133 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f89eff-4a27-4045-bdc2-7fc34d02a2f9 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:55.561662 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a1f89eff-4a27-4045-bdc2-7fc34d02a2f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:55.620687 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f89eff-4a27-4045-bdc2-7fc34d02a2f9 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:55.624175 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f12ede-c396-4236-a0ed-49cfbe750dc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:55.624769 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f12ede-c396-4236-a0ed-49cfbe750dc9 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:55.625048 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f12ede-c396-4236-a0ed-49cfbe750dc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:55.625321 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f12ede-c396-4236-a0ed-49cfbe750dc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:55.627077 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a7f12ede-c396-4236-a0ed-49cfbe750dc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:55.702485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f12ede-c396-4236-a0ed-49cfbe750dc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:55.703947 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f12ede-c396-4236-a0ed-49cfbe750dc9 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:55.715759 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1414/2827] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:58.083705 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de788425-3890-44f4-94e7-d35f4a19c395 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:58.085185 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-de788425-3890-44f4-94e7-d35f4a19c395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:58.097772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de788425-3890-44f4-94e7-d35f4a19c395 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:58.101287 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834cca7-715a-4ab4-938c-4b11aff9e474 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:58.101879 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834cca7-715a-4ab4-938c-4b11aff9e474 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:58.102157 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834cca7-715a-4ab4-938c-4b11aff9e474 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:58.102464 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834cca7-715a-4ab4-938c-4b11aff9e474 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:58.104125 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-c834cca7-715a-4ab4-938c-4b11aff9e474 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:58.120470 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834cca7-715a-4ab4-938c-4b11aff9e474 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:58.121934 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834cca7-715a-4ab4-938c-4b11aff9e474 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:58.134015 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1414/2828] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:44:05.710451 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=54 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:08.128289 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=31 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:21.742386 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f12ede-c396-4236-a0ed-49cfbe750dc9 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:21.751626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f12ede-c396-4236-a0ed-49cfbe750dc9 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:21.756594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc7e4a-7127-4925-8d58-7ade0f55b7e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:21.758061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc7e4a-7127-4925-8d58-7ade0f55b7e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:21.758715 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc7e4a-7127-4925-8d58-7ade0f55b7e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:21.759139 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc7e4a-7127-4925-8d58-7ade0f55b7e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:21.761090 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-45bc7e4a-7127-4925-8d58-7ade0f55b7e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:21.830925 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc7e4a-7127-4925-8d58-7ade0f55b7e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:21.832341 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc7e4a-7127-4925-8d58-7ade0f55b7e3 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:21.843263 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1415/2829] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:44:27.195015 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834cca7-715a-4ab4-938c-4b11aff9e474 service neutron] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:27.204049 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834cca7-715a-4ab4-938c-4b11aff9e474 service neutron] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:27.207839 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac445ccd-247a-4b21-b538-d10ef05fda36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:27.208456 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac445ccd-247a-4b21-b538-d10ef05fda36 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:27.208736 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac445ccd-247a-4b21-b538-d10ef05fda36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:27.209001 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac445ccd-247a-4b21-b538-d10ef05fda36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:27.211185 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-ac445ccd-247a-4b21-b538-d10ef05fda36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:27.226706 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac445ccd-247a-4b21-b538-d10ef05fda36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:27.228352 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac445ccd-247a-4b21-b538-d10ef05fda36 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:27.258349 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1415/2830] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:44:28.137919 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc7e4a-7127-4925-8d58-7ade0f55b7e3 service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:28.139428 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-45bc7e4a-7127-4925-8d58-7ade0f55b7e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:28.172646 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc7e4a-7127-4925-8d58-7ade0f55b7e3 service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:28.179646 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca8ef8-5557-4768-8fd2-35c44c4ab9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:28.180327 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca8ef8-5557-4768-8fd2-35c44c4ab9b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:28.180749 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca8ef8-5557-4768-8fd2-35c44c4ab9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:28.181072 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca8ef8-5557-4768-8fd2-35c44c4ab9b0 admin admin] PATH_INFO: `/v3/users/aa6ad9c2c1294633a74782c668227886` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:28.183607 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca8ef8-5557-4768-8fd2-35c44c4ab9b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa6ad9c2c1294633a74782c668227886)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:28.185671 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca8ef8-5557-4768-8fd2-35c44c4ab9b0 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:28.256478 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0aca8ef8-5557-4768-8fd2-35c44c4ab9b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa6ad9c2c1294633a74782c668227886'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:28.265625 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0aca8ef8-5557-4768-8fd2-35c44c4ab9b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa6ad9c2c1294633a74782c668227886'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:28.266583 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0aca8ef8-5557-4768-8fd2-35c44c4ab9b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa6ad9c2c1294633a74782c668227886'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:28.286203 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0aca8ef8-5557-4768-8fd2-35c44c4ab9b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa6ad9c2c1294633a74782c668227886'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:28.287038 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0aca8ef8-5557-4768-8fd2-35c44c4ab9b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa6ad9c2c1294633a74782c668227886'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:28.305909 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1416/2831] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:44:28 2022] DELETE /identity/v3/users/aa6ad9c2c1294633a74782c668227886 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:28.495987 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac445ccd-247a-4b21-b538-d10ef05fda36 service neutron] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:28.509360 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac445ccd-247a-4b21-b538-d10ef05fda36 service neutron] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:28.586228 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a06b59-18d9-41fa-a7e1-5b912a0a3903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:28.586934 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a06b59-18d9-41fa-a7e1-5b912a0a3903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:28.587217 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a06b59-18d9-41fa-a7e1-5b912a0a3903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:28.587492 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a06b59-18d9-41fa-a7e1-5b912a0a3903 admin admin] PATH_INFO: `/v3/projects/b683a11c21fe4dd389a006f051576a1b` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:28.588824 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a06b59-18d9-41fa-a7e1-5b912a0a3903 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b683a11c21fe4dd389a006f051576a1b)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:28.590772 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a06b59-18d9-41fa-a7e1-5b912a0a3903 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:28.641174 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-e2a06b59-18d9-41fa-a7e1-5b912a0a3903 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b683a11c21fe4dd389a006f051576a1b', 'request_id': 'req-e2a06b59-18d9-41fa-a7e1-5b912a0a3903'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:28.648907 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-e2a06b59-18d9-41fa-a7e1-5b912a0a3903 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b683a11c21fe4dd389a006f051576a1b', 'request_id': 'req-e2a06b59-18d9-41fa-a7e1-5b912a0a3903'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:28.655379 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-e2a06b59-18d9-41fa-a7e1-5b912a0a3903 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b683a11c21fe4dd389a006f051576a1b', 'request_id': 'req-e2a06b59-18d9-41fa-a7e1-5b912a0a3903'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:28.671249 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-e2a06b59-18d9-41fa-a7e1-5b912a0a3903 admin admin] The token cache is being invalidate because project b683a11c21fe4dd389a006f051576a1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:44:28.671483 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-e2a06b59-18d9-41fa-a7e1-5b912a0a3903 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:28.672145 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-e2a06b59-18d9-41fa-a7e1-5b912a0a3903 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:28.673624 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1416/2832] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:44:28 2022] DELETE /identity/v3/projects/b683a11c21fe4dd389a006f051576a1b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:28.678330 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca8ef8-5557-4768-8fd2-35c44c4ab9b0 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:28.679865 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0aca8ef8-5557-4768-8fd2-35c44c4ab9b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:28.693987 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca8ef8-5557-4768-8fd2-35c44c4ab9b0 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:28.768730 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ef21d-7ae7-4543-94e7-68f24d939abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:28.769288 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ef21d-7ae7-4543-94e7-68f24d939abc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:28.769572 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ef21d-7ae7-4543-94e7-68f24d939abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:28.769847 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ef21d-7ae7-4543-94e7-68f24d939abc admin admin] PATH_INFO: `/v3/users/a477a6b7a6724c40a0efb808edfef2a4` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:28.771065 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ef21d-7ae7-4543-94e7-68f24d939abc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a477a6b7a6724c40a0efb808edfef2a4)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:28.772978 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ef21d-7ae7-4543-94e7-68f24d939abc admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:28.781062 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:28.834599 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-930ef21d-7ae7-4543-94e7-68f24d939abc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a477a6b7a6724c40a0efb808edfef2a4'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:28.843172 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-930ef21d-7ae7-4543-94e7-68f24d939abc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a477a6b7a6724c40a0efb808edfef2a4'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:28.844619 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-930ef21d-7ae7-4543-94e7-68f24d939abc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a477a6b7a6724c40a0efb808edfef2a4'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:28.861832 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-930ef21d-7ae7-4543-94e7-68f24d939abc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a477a6b7a6724c40a0efb808edfef2a4'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:28.861832 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-930ef21d-7ae7-4543-94e7-68f24d939abc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a477a6b7a6724c40a0efb808edfef2a4'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:28.882049 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1417/2833] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:44:28 2022] DELETE /identity/v3/users/a477a6b7a6724c40a0efb808edfef2a4 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:44:29.088821 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a06b59-18d9-41fa-a7e1-5b912a0a3903 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:29.090804 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-e2a06b59-18d9-41fa-a7e1-5b912a0a3903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:29.118421 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a06b59-18d9-41fa-a7e1-5b912a0a3903 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:29.188217 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210f7f7-248d-4d54-97e8-786e87527662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:29.189210 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210f7f7-248d-4d54-97e8-786e87527662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:29.189349 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210f7f7-248d-4d54-97e8-786e87527662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:29.189605 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210f7f7-248d-4d54-97e8-786e87527662 admin admin] PATH_INFO: `/v3/projects/5b4f3cdd63e64925afa07955c7aaf518` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:29.191221 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2210f7f7-248d-4d54-97e8-786e87527662 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b4f3cdd63e64925afa07955c7aaf518)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:29.193149 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2210f7f7-248d-4d54-97e8-786e87527662 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:29.227118 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:29.263648 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-2210f7f7-248d-4d54-97e8-786e87527662 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b4f3cdd63e64925afa07955c7aaf518', 'request_id': 'req-2210f7f7-248d-4d54-97e8-786e87527662'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:29.273697 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-2210f7f7-248d-4d54-97e8-786e87527662 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b4f3cdd63e64925afa07955c7aaf518', 'request_id': 'req-2210f7f7-248d-4d54-97e8-786e87527662'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:29.281851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-2210f7f7-248d-4d54-97e8-786e87527662 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b4f3cdd63e64925afa07955c7aaf518', 'request_id': 'req-2210f7f7-248d-4d54-97e8-786e87527662'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:29.294638 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-2210f7f7-248d-4d54-97e8-786e87527662 admin admin] The token cache is being invalidate because project 5b4f3cdd63e64925afa07955c7aaf518 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:44:29.295032 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-2210f7f7-248d-4d54-97e8-786e87527662 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:29.295850 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-2210f7f7-248d-4d54-97e8-786e87527662 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:29.297728 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1417/2834] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:44:29 2022] DELETE /identity/v3/projects/5b4f3cdd63e64925afa07955c7aaf518 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:44:29.917296 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ef21d-7ae7-4543-94e7-68f24d939abc admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:29.918854 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-930ef21d-7ae7-4543-94e7-68f24d939abc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:29.933714 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ef21d-7ae7-4543-94e7-68f24d939abc admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:29.999557 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f622b-52f9-4449-8fae-e7fd4f998b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:30.000242 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f622b-52f9-4449-8fae-e7fd4f998b70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:30.000532 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f622b-52f9-4449-8fae-e7fd4f998b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:30.000815 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f622b-52f9-4449-8fae-e7fd4f998b70 admin admin] PATH_INFO: `/v3/users/a3059f5115bb4bad837daf9f18752143` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:30.001969 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f622b-52f9-4449-8fae-e7fd4f998b70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3059f5115bb4bad837daf9f18752143)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:30.004945 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f622b-52f9-4449-8fae-e7fd4f998b70 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:30.057381 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-961f622b-52f9-4449-8fae-e7fd4f998b70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3059f5115bb4bad837daf9f18752143'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:30.066449 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-961f622b-52f9-4449-8fae-e7fd4f998b70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3059f5115bb4bad837daf9f18752143'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:30.067075 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-961f622b-52f9-4449-8fae-e7fd4f998b70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3059f5115bb4bad837daf9f18752143'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:30.079033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-961f622b-52f9-4449-8fae-e7fd4f998b70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3059f5115bb4bad837daf9f18752143'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:30.079618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-961f622b-52f9-4449-8fae-e7fd4f998b70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3059f5115bb4bad837daf9f18752143'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:30.092770 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1418/2835] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:44:29 2022] DELETE /identity/v3/users/a3059f5115bb4bad837daf9f18752143 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:30.272605 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210f7f7-248d-4d54-97e8-786e87527662 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:30.274101 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-2210f7f7-248d-4d54-97e8-786e87527662 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:30.289015 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210f7f7-248d-4d54-97e8-786e87527662 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:30.349205 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31cc110-3445-478c-9254-2f05b8d73216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:30.349748 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cc110-3445-478c-9254-2f05b8d73216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:30.350030 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cc110-3445-478c-9254-2f05b8d73216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:30.350339 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cc110-3445-478c-9254-2f05b8d73216 admin admin] PATH_INFO: `/v3/projects/4b6bf55e6c5e47f3895e135c4d6dc3ff` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:30.351483 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31cc110-3445-478c-9254-2f05b8d73216 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b6bf55e6c5e47f3895e135c4d6dc3ff)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:30.353396 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31cc110-3445-478c-9254-2f05b8d73216 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:30.406207 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c31cc110-3445-478c-9254-2f05b8d73216 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b6bf55e6c5e47f3895e135c4d6dc3ff', 'request_id': 'req-c31cc110-3445-478c-9254-2f05b8d73216'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:30.413898 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c31cc110-3445-478c-9254-2f05b8d73216 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b6bf55e6c5e47f3895e135c4d6dc3ff', 'request_id': 'req-c31cc110-3445-478c-9254-2f05b8d73216'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:30.420643 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c31cc110-3445-478c-9254-2f05b8d73216 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b6bf55e6c5e47f3895e135c4d6dc3ff', 'request_id': 'req-c31cc110-3445-478c-9254-2f05b8d73216'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:30.431436 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c31cc110-3445-478c-9254-2f05b8d73216 admin admin] The token cache is being invalidate because project 4b6bf55e6c5e47f3895e135c4d6dc3ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:44:30.431729 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c31cc110-3445-478c-9254-2f05b8d73216 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:30.432362 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-c31cc110-3445-478c-9254-2f05b8d73216 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:30.433850 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1418/2836] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:44:30 2022] DELETE /identity/v3/projects/4b6bf55e6c5e47f3895e135c4d6dc3ff => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:30.438062 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f622b-52f9-4449-8fae-e7fd4f998b70 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:30.439697 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-961f622b-52f9-4449-8fae-e7fd4f998b70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:30.449463 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:30.453763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f622b-52f9-4449-8fae-e7fd4f998b70 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:30.515215 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1594d6c-a0bf-4d7a-bd75-c7e70f2dbce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:30.515785 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1594d6c-a0bf-4d7a-bd75-c7e70f2dbce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:30.516068 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1594d6c-a0bf-4d7a-bd75-c7e70f2dbce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:30.516344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1594d6c-a0bf-4d7a-bd75-c7e70f2dbce8 admin admin] PATH_INFO: `/v3/users/41e9711b60e4413b9df5575391dae5cb` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:30.517726 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1594d6c-a0bf-4d7a-bd75-c7e70f2dbce8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41e9711b60e4413b9df5575391dae5cb)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:30.520248 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1594d6c-a0bf-4d7a-bd75-c7e70f2dbce8 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:30.571810 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f1594d6c-a0bf-4d7a-bd75-c7e70f2dbce8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41e9711b60e4413b9df5575391dae5cb'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:30.582778 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f1594d6c-a0bf-4d7a-bd75-c7e70f2dbce8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41e9711b60e4413b9df5575391dae5cb'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:30.583400 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f1594d6c-a0bf-4d7a-bd75-c7e70f2dbce8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41e9711b60e4413b9df5575391dae5cb'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:30.597346 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f1594d6c-a0bf-4d7a-bd75-c7e70f2dbce8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41e9711b60e4413b9df5575391dae5cb'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:30.598046 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-f1594d6c-a0bf-4d7a-bd75-c7e70f2dbce8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41e9711b60e4413b9df5575391dae5cb'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:30.610390 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1419/2837] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:44:30 2022] DELETE /identity/v3/users/41e9711b60e4413b9df5575391dae5cb => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:30.792964 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31cc110-3445-478c-9254-2f05b8d73216 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:30.794485 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-c31cc110-3445-478c-9254-2f05b8d73216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:30.808142 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31cc110-3445-478c-9254-2f05b8d73216 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:30.842719 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:30.871172 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a417ba-8f35-4bbe-a743-8980b721573a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:30.871724 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a417ba-8f35-4bbe-a743-8980b721573a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:30.872005 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a417ba-8f35-4bbe-a743-8980b721573a admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:30.872275 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a417ba-8f35-4bbe-a743-8980b721573a admin admin] PATH_INFO: `/v3/projects/38a410afaff64327b9140b767b4ae48c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:30.873422 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a417ba-8f35-4bbe-a743-8980b721573a admin admin] RBAC: Authorizing `identity:delete_project(project_id=38a410afaff64327b9140b767b4ae48c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:30.875422 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a417ba-8f35-4bbe-a743-8980b721573a admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:30.923550 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-24a417ba-8f35-4bbe-a743-8980b721573a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38a410afaff64327b9140b767b4ae48c', 'request_id': 'req-24a417ba-8f35-4bbe-a743-8980b721573a'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:30.931614 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-24a417ba-8f35-4bbe-a743-8980b721573a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38a410afaff64327b9140b767b4ae48c', 'request_id': 'req-24a417ba-8f35-4bbe-a743-8980b721573a'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:30.938815 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-24a417ba-8f35-4bbe-a743-8980b721573a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38a410afaff64327b9140b767b4ae48c', 'request_id': 'req-24a417ba-8f35-4bbe-a743-8980b721573a'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:30.950414 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-24a417ba-8f35-4bbe-a743-8980b721573a admin admin] The token cache is being invalidate because project 38a410afaff64327b9140b767b4ae48c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:44:30.950702 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-24a417ba-8f35-4bbe-a743-8980b721573a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:30.951352 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-24a417ba-8f35-4bbe-a743-8980b721573a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:30.952883 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1419/2838] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:44:30 2022] DELETE /identity/v3/projects/38a410afaff64327b9140b767b4ae48c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:40.606809 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=51,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:40.947236 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=34,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:49.305984 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1594d6c-a0bf-4d7a-bd75-c7e70f2dbce8 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:49.307569 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-f1594d6c-a0bf-4d7a-bd75-c7e70f2dbce8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:49.413998 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1594d6c-a0bf-4d7a-bd75-c7e70f2dbce8 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:49.501335 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445de1b6-0b5e-4a2d-ae5b-e43c43bc38bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '278e6d6aed414c758fe3e9f4df6bf81e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:49.502556 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445de1b6-0b5e-4a2d-ae5b-e43c43bc38bf service nova] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:49.502839 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445de1b6-0b5e-4a2d-ae5b-e43c43bc38bf service nova] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:49.503112 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445de1b6-0b5e-4a2d-ae5b-e43c43bc38bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:49.504986 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-445de1b6-0b5e-4a2d-ae5b-e43c43bc38bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:49.522006 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445de1b6-0b5e-4a2d-ae5b-e43c43bc38bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:49.524468 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445de1b6-0b5e-4a2d-ae5b-e43c43bc38bf service nova] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:49.550586 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1420/2839] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:44:52.213848 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c72b1-5755-4aae-a1d4-51efaf98728d None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:52.214409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c72b1-5755-4aae-a1d4-51efaf98728d None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:52.214409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c72b1-5755-4aae-a1d4-51efaf98728d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:52.234093 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-ac7c72b1-5755-4aae-a1d4-51efaf98728d None None] MFA Rules not processed for user `a237b400429f4200909581e3c5e0cbfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:52.235326 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-ac7c72b1-5755-4aae-a1d4-51efaf98728d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:52.289399 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1420/2840] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:44:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:52.920012 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445de1b6-0b5e-4a2d-ae5b-e43c43bc38bf service nova] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:52.921432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-445de1b6-0b5e-4a2d-ae5b-e43c43bc38bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:52.997084 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445de1b6-0b5e-4a2d-ae5b-e43c43bc38bf service nova] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:53.002565 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042fe25-a2bd-4e28-af95-c0f6cdc6b51a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ddfb151bbf844efae73ec1990e7eaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:53.003173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042fe25-a2bd-4e28-af95-c0f6cdc6b51a service placement] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:53.003461 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042fe25-a2bd-4e28-af95-c0f6cdc6b51a service placement] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:53.003736 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042fe25-a2bd-4e28-af95-c0f6cdc6b51a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:53.005809 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7042fe25-a2bd-4e28-af95-c0f6cdc6b51a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:53.018946 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7042fe25-a2bd-4e28-af95-c0f6cdc6b51a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:53.020513 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7042fe25-a2bd-4e28-af95-c0f6cdc6b51a service placement] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:53.030299 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1421/2841] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:44:54.850741 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c72b1-5755-4aae-a1d4-51efaf98728d None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:54.852372 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-ac7c72b1-5755-4aae-a1d4-51efaf98728d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:54.917323 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c72b1-5755-4aae-a1d4-51efaf98728d None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:54.921336 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6342a-4297-4fe5-8808-a7001894b4e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:54.922160 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6342a-4297-4fe5-8808-a7001894b4e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:54.922569 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6342a-4297-4fe5-8808-a7001894b4e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:54.922835 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6342a-4297-4fe5-8808-a7001894b4e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:54.924833 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-dbe6342a-4297-4fe5-8808-a7001894b4e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:54.936555 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe6342a-4297-4fe5-8808-a7001894b4e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:54.938347 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe6342a-4297-4fe5-8808-a7001894b4e4 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:54.947268 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1421/2842] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:03.026065 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=94 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:04.943499 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=51 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:00.313206 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042fe25-a2bd-4e28-af95-c0f6cdc6b51a service placement] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:00.315122 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7042fe25-a2bd-4e28-af95-c0f6cdc6b51a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:00.389763 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042fe25-a2bd-4e28-af95-c0f6cdc6b51a service placement] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:00.398432 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828bd09d-c97d-4b32-ab60-302aac711670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:00.399113 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828bd09d-c97d-4b32-ab60-302aac711670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:00.399396 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828bd09d-c97d-4b32-ab60-302aac711670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:00.399671 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828bd09d-c97d-4b32-ab60-302aac711670 admin admin] PATH_INFO: `/v3/users/a237b400429f4200909581e3c5e0cbfb` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:00.400824 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828bd09d-c97d-4b32-ab60-302aac711670 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a237b400429f4200909581e3c5e0cbfb)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:00.402824 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828bd09d-c97d-4b32-ab60-302aac711670 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:00.459638 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-828bd09d-c97d-4b32-ab60-302aac711670 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a237b400429f4200909581e3c5e0cbfb'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:00.468591 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-828bd09d-c97d-4b32-ab60-302aac711670 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a237b400429f4200909581e3c5e0cbfb'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:00.469416 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-828bd09d-c97d-4b32-ab60-302aac711670 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a237b400429f4200909581e3c5e0cbfb'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:00.483561 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-828bd09d-c97d-4b32-ab60-302aac711670 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a237b400429f4200909581e3c5e0cbfb'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:00.484310 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-828bd09d-c97d-4b32-ab60-302aac711670 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a237b400429f4200909581e3c5e0cbfb'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:00.497916 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1422/2843] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:46:00 2022] DELETE /identity/v3/users/a237b400429f4200909581e3c5e0cbfb => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:00.680619 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6342a-4297-4fe5-8808-a7001894b4e4 service neutron] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:00.699690 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6342a-4297-4fe5-8808-a7001894b4e4 service neutron] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:00.759304 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f083380-8ea4-4d97-ac5b-98f148c2fc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:00.759944 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f083380-8ea4-4d97-ac5b-98f148c2fc20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:00.760223 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f083380-8ea4-4d97-ac5b-98f148c2fc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:00.760497 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f083380-8ea4-4d97-ac5b-98f148c2fc20 admin admin] PATH_INFO: `/v3/projects/9b9aa20e1ffb453d9ee13dd6ae2c25ef` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:00.761662 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f083380-8ea4-4d97-ac5b-98f148c2fc20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b9aa20e1ffb453d9ee13dd6ae2c25ef)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:00.763645 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f083380-8ea4-4d97-ac5b-98f148c2fc20 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:00.826577 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-8f083380-8ea4-4d97-ac5b-98f148c2fc20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b9aa20e1ffb453d9ee13dd6ae2c25ef', 'request_id': 'req-8f083380-8ea4-4d97-ac5b-98f148c2fc20'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:00.835205 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-8f083380-8ea4-4d97-ac5b-98f148c2fc20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b9aa20e1ffb453d9ee13dd6ae2c25ef', 'request_id': 'req-8f083380-8ea4-4d97-ac5b-98f148c2fc20'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:00.845004 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-8f083380-8ea4-4d97-ac5b-98f148c2fc20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b9aa20e1ffb453d9ee13dd6ae2c25ef', 'request_id': 'req-8f083380-8ea4-4d97-ac5b-98f148c2fc20'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:00.856409 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-8f083380-8ea4-4d97-ac5b-98f148c2fc20 admin admin] The token cache is being invalidate because project 9b9aa20e1ffb453d9ee13dd6ae2c25ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:00.856701 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-8f083380-8ea4-4d97-ac5b-98f148c2fc20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:00.857350 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-8f083380-8ea4-4d97-ac5b-98f148c2fc20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:00.859082 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1422/2844] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:00 2022] DELETE /identity/v3/projects/9b9aa20e1ffb453d9ee13dd6ae2c25ef => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:00.864456 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fcb62-db34-4cd4-ba81-26b2340e07ac None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:00.864749 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fcb62-db34-4cd4-ba81-26b2340e07ac None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:00.865028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fcb62-db34-4cd4-ba81-26b2340e07ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:00.914789 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-db7fcb62-db34-4cd4-ba81-26b2340e07ac None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:00.916020 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-db7fcb62-db34-4cd4-ba81-26b2340e07ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:00.964805 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:00.980819 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1423/2845] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:46:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:00.989094 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1d86a-631a-401c-93ba-20ca8e290dbc None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:00.989388 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1d86a-631a-401c-93ba-20ca8e290dbc None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:00.989672 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1d86a-631a-401c-93ba-20ca8e290dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:01.011098 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-85b1d86a-631a-401c-93ba-20ca8e290dbc None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:01.012209 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-85b1d86a-631a-401c-93ba-20ca8e290dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:01.022846 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1423/2846] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:46:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:01.027191 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7fcb62-db34-4cd4-ba81-26b2340e07ac None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:01.037657 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7fcb62-db34-4cd4-ba81-26b2340e07ac None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:01.042329 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df31fb2-410d-45b6-a74a-cc36f69051d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:01.042949 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df31fb2-410d-45b6-a74a-cc36f69051d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:01.043233 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df31fb2-410d-45b6-a74a-cc36f69051d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:01.043510 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df31fb2-410d-45b6-a74a-cc36f69051d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:01.044204 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df31fb2-410d-45b6-a74a-cc36f69051d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:46:01.044516 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df31fb2-410d-45b6-a74a-cc36f69051d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:01.046126 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df31fb2-410d-45b6-a74a-cc36f69051d3 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:01.060066 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1424/2847] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:46:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:46:01.064429 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b1d86a-631a-401c-93ba-20ca8e290dbc None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:01.077342 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b1d86a-631a-401c-93ba-20ca8e290dbc None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:01.088048 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee380f6-cc9c-441a-ac04-615961bbbb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:01.088677 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee380f6-cc9c-441a-ac04-615961bbbb78 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:01.088969 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee380f6-cc9c-441a-ac04-615961bbbb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:01.089245 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee380f6-cc9c-441a-ac04-615961bbbb78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:01.089910 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee380f6-cc9c-441a-ac04-615961bbbb78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:01.091888 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee380f6-cc9c-441a-ac04-615961bbbb78 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:01.134433 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1424/2848] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:46:01 2022] POST /identity/v3/projects => generated 405 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:01.140195 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df31fb2-410d-45b6-a74a-cc36f69051d3 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:01.151134 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df31fb2-410d-45b6-a74a-cc36f69051d3 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:01.212237 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce501e-f584-4ee4-b80a-0c45b0499fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:01.212890 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce501e-f584-4ee4-b80a-0c45b0499fa4 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:01.213173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce501e-f584-4ee4-b80a-0c45b0499fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:01.213448 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce501e-f584-4ee4-b80a-0c45b0499fa4 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:01.214084 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce501e-f584-4ee4-b80a-0c45b0499fa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:01.216044 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce501e-f584-4ee4-b80a-0c45b0499fa4 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:01.239614 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1425/2849] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:46:01 2022] POST /identity/v3/users => generated 303 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:01.243713 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee380f6-cc9c-441a-ac04-615961bbbb78 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:01.256718 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee380f6-cc9c-441a-ac04-615961bbbb78 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:01.261408 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf062192-6733-4efb-8532-3591f5876f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:01.262019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf062192-6733-4efb-8532-3591f5876f38 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:01.262366 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf062192-6733-4efb-8532-3591f5876f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:01.262603 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf062192-6733-4efb-8532-3591f5876f38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:01.263248 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf062192-6733-4efb-8532-3591f5876f38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:01.264863 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf062192-6733-4efb-8532-3591f5876f38 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:01.275425 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1425/2850] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:01.279610 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce501e-f584-4ee4-b80a-0c45b0499fa4 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:01.290276 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce501e-f584-4ee4-b80a-0c45b0499fa4 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:01.295007 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed4846-31c1-4611-b363-0810ddc56301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:01.295618 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed4846-31c1-4611-b363-0810ddc56301 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:01.295903 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed4846-31c1-4611-b363-0810ddc56301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:01.296177 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed4846-31c1-4611-b363-0810ddc56301 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:01.296819 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed4846-31c1-4611-b363-0810ddc56301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:01.298446 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed4846-31c1-4611-b363-0810ddc56301 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:01.310104 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1426/2851] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:46:01.314421 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf062192-6733-4efb-8532-3591f5876f38 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:01.324923 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf062192-6733-4efb-8532-3591f5876f38 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:01.334954 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2069f8-122d-4956-b3f9-b797253be65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:01.334954 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2069f8-122d-4956-b3f9-b797253be65d admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:01.334954 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2069f8-122d-4956-b3f9-b797253be65d admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:01.334954 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2069f8-122d-4956-b3f9-b797253be65d admin admin] PATH_INFO: `/v3/projects/6391215b14954c679300b94a07ae4c55/users/982aec8590104cee94d8e3ff6d1e2f2f/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:01.348264 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2069f8-122d-4956-b3f9-b797253be65d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6391215b14954c679300b94a07ae4c55, user_id=982aec8590104cee94d8e3ff6d1e2f2f, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:01.351038 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2069f8-122d-4956-b3f9-b797253be65d admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:01.368452 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1426/2852] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:01 2022] PUT /identity/v3/projects/6391215b14954c679300b94a07ae4c55/users/982aec8590104cee94d8e3ff6d1e2f2f/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:01.373625 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed48ab9-cdd5-4431-a4be-d4a0db3a5664 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:01.374159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed48ab9-cdd5-4431-a4be-d4a0db3a5664 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:01.374159 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed48ab9-cdd5-4431-a4be-d4a0db3a5664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:01.394899 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-bed48ab9-cdd5-4431-a4be-d4a0db3a5664 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:01.396353 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-bed48ab9-cdd5-4431-a4be-d4a0db3a5664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:01.457341 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1427/2853] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:46:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:01.467291 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2069f8-122d-4956-b3f9-b797253be65d admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:01.469013 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-8e2069f8-122d-4956-b3f9-b797253be65d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:01.491396 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:01.539490 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2069f8-122d-4956-b3f9-b797253be65d admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:01.544358 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fef52a-99c5-4efc-a14a-264ba05ae202 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:01.544975 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fef52a-99c5-4efc-a14a-264ba05ae202 service neutron] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:01.545259 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fef52a-99c5-4efc-a14a-264ba05ae202 service neutron] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:01.545531 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fef52a-99c5-4efc-a14a-264ba05ae202 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:01.555600 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fef52a-99c5-4efc-a14a-264ba05ae202 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:01.557075 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fef52a-99c5-4efc-a14a-264ba05ae202 service neutron] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:01.567586 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1427/2854] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:02.226186 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8913f-d6be-4ba3-91dc-3e736a003293 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:02.226497 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8913f-d6be-4ba3-91dc-3e736a003293 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:02.226772 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8913f-d6be-4ba3-91dc-3e736a003293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:02.257610 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-29a8913f-d6be-4ba3-91dc-3e736a003293 None None] MFA Rules not processed for user `982aec8590104cee94d8e3ff6d1e2f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:02.258900 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-29a8913f-d6be-4ba3-91dc-3e736a003293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:02.320649 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:02.336996 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1428/2855] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:46:02 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:02.347115 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fef52a-99c5-4efc-a14a-264ba05ae202 service neutron] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:02.358302 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fef52a-99c5-4efc-a14a-264ba05ae202 service neutron] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:02.369300 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74fbf4a-1d5d-40e7-8bfe-d063411eb1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:02.370125 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fbf4a-1d5d-40e7-8bfe-d063411eb1b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:02.370879 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fbf4a-1d5d-40e7-8bfe-d063411eb1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:02.371006 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fbf4a-1d5d-40e7-8bfe-d063411eb1b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:02.372272 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74fbf4a-1d5d-40e7-8bfe-d063411eb1b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:02.374849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74fbf4a-1d5d-40e7-8bfe-d063411eb1b9 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:02.415238 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1428/2856] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:46:02 2022] POST /identity/v3/projects => generated 405 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:02.419442 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a8913f-d6be-4ba3-91dc-3e736a003293 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:02.430898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a8913f-d6be-4ba3-91dc-3e736a003293 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:02.488752 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592abd1c-e06d-4ff0-af4c-ab48d29d5f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:02.489365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592abd1c-e06d-4ff0-af4c-ab48d29d5f87 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:02.489645 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592abd1c-e06d-4ff0-af4c-ab48d29d5f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:02.489918 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592abd1c-e06d-4ff0-af4c-ab48d29d5f87 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:02.490585 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592abd1c-e06d-4ff0-af4c-ab48d29d5f87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:02.492509 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592abd1c-e06d-4ff0-af4c-ab48d29d5f87 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:02.513779 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1429/2857] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:46:02 2022] POST /identity/v3/users => generated 309 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:02.517801 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74fbf4a-1d5d-40e7-8bfe-d063411eb1b9 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:02.528276 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74fbf4a-1d5d-40e7-8bfe-d063411eb1b9 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:02.538438 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd549b93-0755-4d56-87f7-db8ba40b096d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:02.539046 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd549b93-0755-4d56-87f7-db8ba40b096d admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:02.539323 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd549b93-0755-4d56-87f7-db8ba40b096d admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:02.539591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd549b93-0755-4d56-87f7-db8ba40b096d admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:02.540227 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd549b93-0755-4d56-87f7-db8ba40b096d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:02.541829 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd549b93-0755-4d56-87f7-db8ba40b096d admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:02.553620 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1429/2858] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:02.557872 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592abd1c-e06d-4ff0-af4c-ab48d29d5f87 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:02.568632 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592abd1c-e06d-4ff0-af4c-ab48d29d5f87 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:02.573349 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c846be-50fd-4067-8882-b0aff4e91d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:02.574009 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c846be-50fd-4067-8882-b0aff4e91d20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:02.574338 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c846be-50fd-4067-8882-b0aff4e91d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:02.574605 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c846be-50fd-4067-8882-b0aff4e91d20 admin admin] PATH_INFO: `/v3/projects/862feca035a940f188289f335468b933/users/73e3a0ad1f7a4fd2a3c0badf4cc92dd7/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:02.590511 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c846be-50fd-4067-8882-b0aff4e91d20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=862feca035a940f188289f335468b933, user_id=73e3a0ad1f7a4fd2a3c0badf4cc92dd7, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:02.593169 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c846be-50fd-4067-8882-b0aff4e91d20 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:02.607559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1430/2859] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:02 2022] PUT /identity/v3/projects/862feca035a940f188289f335468b933/users/73e3a0ad1f7a4fd2a3c0badf4cc92dd7/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:02.612406 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd549b93-0755-4d56-87f7-db8ba40b096d admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:02.626371 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd549b93-0755-4d56-87f7-db8ba40b096d admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:02.683101 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ec7be-3439-4776-a93e-61e5c947d15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:02.683624 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ec7be-3439-4776-a93e-61e5c947d15c admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:02.683901 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ec7be-3439-4776-a93e-61e5c947d15c admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:02.684171 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ec7be-3439-4776-a93e-61e5c947d15c admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:02.684808 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ec7be-3439-4776-a93e-61e5c947d15c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:02.686447 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ec7be-3439-4776-a93e-61e5c947d15c admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:02.697300 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1430/2860] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:02.702365 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c846be-50fd-4067-8882-b0aff4e91d20 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:02.720285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c846be-50fd-4067-8882-b0aff4e91d20 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:02.725751 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff86e7-b0a1-4ccd-8edb-74b7d39e35f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:02.726474 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff86e7-b0a1-4ccd-8edb-74b7d39e35f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:02.726752 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff86e7-b0a1-4ccd-8edb-74b7d39e35f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:02.727025 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff86e7-b0a1-4ccd-8edb-74b7d39e35f9 admin admin] PATH_INFO: `/v3/projects/862feca035a940f188289f335468b933/users/73e3a0ad1f7a4fd2a3c0badf4cc92dd7/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:02.728946 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff86e7-b0a1-4ccd-8edb-74b7d39e35f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=862feca035a940f188289f335468b933, user_id=73e3a0ad1f7a4fd2a3c0badf4cc92dd7, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:02.731603 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff86e7-b0a1-4ccd-8edb-74b7d39e35f9 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:02.742901 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1431/2861] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:02 2022] PUT /identity/v3/projects/862feca035a940f188289f335468b933/users/73e3a0ad1f7a4fd2a3c0badf4cc92dd7/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:03.437939 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a0c5c-4604-41a3-8999-0580f34d954b None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:03.438708 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a0c5c-4604-41a3-8999-0580f34d954b None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:03.438708 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a0c5c-4604-41a3-8999-0580f34d954b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:03.470692 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-4f6a0c5c-4604-41a3-8999-0580f34d954b None None] MFA Rules not processed for user `73e3a0ad1f7a4fd2a3c0badf4cc92dd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:03.471964 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-4f6a0c5c-4604-41a3-8999-0580f34d954b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:03.552723 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1431/2862] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:46:03 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:04.793868 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff86e7-b0a1-4ccd-8edb-74b7d39e35f9 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:04.795787 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-03ff86e7-b0a1-4ccd-8edb-74b7d39e35f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:04.852023 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a0c5c-4604-41a3-8999-0580f34d954b None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:04.866029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:04.881493 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff86e7-b0a1-4ccd-8edb-74b7d39e35f9 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:04.886693 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e9ae0-f968-4afa-a31b-e5365c9dcd47 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ddfb151bbf844efae73ec1990e7eaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:04.887319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e9ae0-f968-4afa-a31b-e5365c9dcd47 service placement] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:04.887602 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e9ae0-f968-4afa-a31b-e5365c9dcd47 service placement] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:04.887877 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e9ae0-f968-4afa-a31b-e5365c9dcd47 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:04.889732 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-e48e9ae0-f968-4afa-a31b-e5365c9dcd47 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:04.901931 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e9ae0-f968-4afa-a31b-e5365c9dcd47 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:04.903571 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e9ae0-f968-4afa-a31b-e5365c9dcd47 service placement] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:04.913966 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1432/2863] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:04.930763 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a0c5c-4604-41a3-8999-0580f34d954b None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:04.935598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07413623-e4a5-4b19-802e-8c201ab64132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:04.936274 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07413623-e4a5-4b19-802e-8c201ab64132 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:04.936553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07413623-e4a5-4b19-802e-8c201ab64132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:04.936826 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07413623-e4a5-4b19-802e-8c201ab64132 admin admin] PATH_INFO: `/v3/users/982aec8590104cee94d8e3ff6d1e2f2f` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:04.937998 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07413623-e4a5-4b19-802e-8c201ab64132 admin admin] RBAC: Authorizing `identity:delete_user(user_id=982aec8590104cee94d8e3ff6d1e2f2f)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:04.940566 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07413623-e4a5-4b19-802e-8c201ab64132 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:04.985356 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-07413623-e4a5-4b19-802e-8c201ab64132 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '982aec8590104cee94d8e3ff6d1e2f2f'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:04.985999 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-07413623-e4a5-4b19-802e-8c201ab64132 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '982aec8590104cee94d8e3ff6d1e2f2f'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:04.994589 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-07413623-e4a5-4b19-802e-8c201ab64132 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '982aec8590104cee94d8e3ff6d1e2f2f'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:04.995199 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-07413623-e4a5-4b19-802e-8c201ab64132 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '982aec8590104cee94d8e3ff6d1e2f2f'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:05.008028 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-07413623-e4a5-4b19-802e-8c201ab64132 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '982aec8590104cee94d8e3ff6d1e2f2f'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:05.021693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1432/2864] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:46:04 2022] DELETE /identity/v3/users/982aec8590104cee94d8e3ff6d1e2f2f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:46:05.207660 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e9ae0-f968-4afa-a31b-e5365c9dcd47 service placement] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:05.219197 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e9ae0-f968-4afa-a31b-e5365c9dcd47 service placement] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:05.250895 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:05.277629 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7162a52b-06cc-4613-b17c-4ddccee4466f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:05.278315 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162a52b-06cc-4613-b17c-4ddccee4466f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:05.278604 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162a52b-06cc-4613-b17c-4ddccee4466f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:05.278880 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162a52b-06cc-4613-b17c-4ddccee4466f admin admin] PATH_INFO: `/v3/projects/6391215b14954c679300b94a07ae4c55` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:05.280016 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7162a52b-06cc-4613-b17c-4ddccee4466f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6391215b14954c679300b94a07ae4c55)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:05.281907 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7162a52b-06cc-4613-b17c-4ddccee4466f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:05.331253 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7162a52b-06cc-4613-b17c-4ddccee4466f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6391215b14954c679300b94a07ae4c55', 'request_id': 'req-7162a52b-06cc-4613-b17c-4ddccee4466f'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:05.337594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7162a52b-06cc-4613-b17c-4ddccee4466f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6391215b14954c679300b94a07ae4c55', 'request_id': 'req-7162a52b-06cc-4613-b17c-4ddccee4466f'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:05.345721 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7162a52b-06cc-4613-b17c-4ddccee4466f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6391215b14954c679300b94a07ae4c55', 'request_id': 'req-7162a52b-06cc-4613-b17c-4ddccee4466f'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:05.357972 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7162a52b-06cc-4613-b17c-4ddccee4466f admin admin] The token cache is being invalidate because project 6391215b14954c679300b94a07ae4c55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:05.358299 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7162a52b-06cc-4613-b17c-4ddccee4466f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:05.358950 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-7162a52b-06cc-4613-b17c-4ddccee4466f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:05.360428 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1433/2865] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:05 2022] DELETE /identity/v3/projects/6391215b14954c679300b94a07ae4c55 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:05.364395 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07413623-e4a5-4b19-802e-8c201ab64132 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:05.365808 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-07413623-e4a5-4b19-802e-8c201ab64132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:05.378803 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07413623-e4a5-4b19-802e-8c201ab64132 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:05.435357 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72816d27-62ae-4b3d-90c4-65592836c346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:05.436032 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72816d27-62ae-4b3d-90c4-65592836c346 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:05.436313 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72816d27-62ae-4b3d-90c4-65592836c346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:05.436591 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72816d27-62ae-4b3d-90c4-65592836c346 admin admin] PATH_INFO: `/v3/users/73e3a0ad1f7a4fd2a3c0badf4cc92dd7` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:05.437741 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72816d27-62ae-4b3d-90c4-65592836c346 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73e3a0ad1f7a4fd2a3c0badf4cc92dd7)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:05.439715 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72816d27-62ae-4b3d-90c4-65592836c346 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:05.477622 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:05.492964 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-72816d27-62ae-4b3d-90c4-65592836c346 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73e3a0ad1f7a4fd2a3c0badf4cc92dd7'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:05.493656 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-72816d27-62ae-4b3d-90c4-65592836c346 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73e3a0ad1f7a4fd2a3c0badf4cc92dd7'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:05.501347 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-72816d27-62ae-4b3d-90c4-65592836c346 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73e3a0ad1f7a4fd2a3c0badf4cc92dd7'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:05.502026 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-72816d27-62ae-4b3d-90c4-65592836c346 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73e3a0ad1f7a4fd2a3c0badf4cc92dd7'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:05.514313 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-72816d27-62ae-4b3d-90c4-65592836c346 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73e3a0ad1f7a4fd2a3c0badf4cc92dd7'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:05.526903 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1433/2866] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:46:05 2022] DELETE /identity/v3/users/73e3a0ad1f7a4fd2a3c0badf4cc92dd7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:05.698534 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7162a52b-06cc-4613-b17c-4ddccee4466f admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:05.700044 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-7162a52b-06cc-4613-b17c-4ddccee4466f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:05.713162 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7162a52b-06cc-4613-b17c-4ddccee4466f admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:05.775957 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6876c82-afcf-4aa7-aa58-97608cc11fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:05.776596 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6876c82-afcf-4aa7-aa58-97608cc11fea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:05.776875 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6876c82-afcf-4aa7-aa58-97608cc11fea admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:05.777146 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6876c82-afcf-4aa7-aa58-97608cc11fea admin admin] PATH_INFO: `/v3/projects/862feca035a940f188289f335468b933` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:05.778267 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6876c82-afcf-4aa7-aa58-97608cc11fea admin admin] RBAC: Authorizing `identity:delete_project(project_id=862feca035a940f188289f335468b933)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:05.780182 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6876c82-afcf-4aa7-aa58-97608cc11fea admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:05.831331 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b6876c82-afcf-4aa7-aa58-97608cc11fea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '862feca035a940f188289f335468b933', 'request_id': 'req-b6876c82-afcf-4aa7-aa58-97608cc11fea'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:05.837586 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b6876c82-afcf-4aa7-aa58-97608cc11fea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '862feca035a940f188289f335468b933', 'request_id': 'req-b6876c82-afcf-4aa7-aa58-97608cc11fea'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:05.844546 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b6876c82-afcf-4aa7-aa58-97608cc11fea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '862feca035a940f188289f335468b933', 'request_id': 'req-b6876c82-afcf-4aa7-aa58-97608cc11fea'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:05.848820 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:05.857207 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b6876c82-afcf-4aa7-aa58-97608cc11fea admin admin] The token cache is being invalidate because project 862feca035a940f188289f335468b933 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81551) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:05.857498 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b6876c82-afcf-4aa7-aa58-97608cc11fea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:05.858228 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-b6876c82-afcf-4aa7-aa58-97608cc11fea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:05.860001 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1434/2867] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:05 2022] DELETE /identity/v3/projects/862feca035a940f188289f335468b933 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:05.865117 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d825c0-17d5-4c46-98ea-505442ae2d86 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:05.865627 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d825c0-17d5-4c46-98ea-505442ae2d86 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:05.865748 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d825c0-17d5-4c46-98ea-505442ae2d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:05.885686 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-15d825c0-17d5-4c46-98ea-505442ae2d86 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:05.886945 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-15d825c0-17d5-4c46-98ea-505442ae2d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:05.945957 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1434/2868] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:46:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:05.953684 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6f144-82dc-4ae3-93d2-3a799d7276bd None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:05.953969 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6f144-82dc-4ae3-93d2-3a799d7276bd None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:05.954276 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6f144-82dc-4ae3-93d2-3a799d7276bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:05.975709 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-d4f6f144-82dc-4ae3-93d2-3a799d7276bd None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:05.976948 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-d4f6f144-82dc-4ae3-93d2-3a799d7276bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:05.984725 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1435/2869] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:46:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:05.989061 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d825c0-17d5-4c46-98ea-505442ae2d86 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:05.999162 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d825c0-17d5-4c46-98ea-505442ae2d86 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:06.004273 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcdb0fc-c499-4a14-8ce4-3e7a699539a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:06.004881 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcdb0fc-c499-4a14-8ce4-3e7a699539a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:06.005167 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcdb0fc-c499-4a14-8ce4-3e7a699539a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:06.005446 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcdb0fc-c499-4a14-8ce4-3e7a699539a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:06.006126 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcdb0fc-c499-4a14-8ce4-3e7a699539a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81552) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:46:06.006462 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcdb0fc-c499-4a14-8ce4-3e7a699539a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:06.008077 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcdb0fc-c499-4a14-8ce4-3e7a699539a0 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:06.022326 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1435/2870] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:46:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:46:06.026875 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f6f144-82dc-4ae3-93d2-3a799d7276bd None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:06.036141 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f6f144-82dc-4ae3-93d2-3a799d7276bd None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:06.040738 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0cc9fe-db58-4100-bc16-809daa3e5dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:06.041362 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0cc9fe-db58-4100-bc16-809daa3e5dbd admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:06.041639 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0cc9fe-db58-4100-bc16-809daa3e5dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:06.041908 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0cc9fe-db58-4100-bc16-809daa3e5dbd admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:06.042574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0cc9fe-db58-4100-bc16-809daa3e5dbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:06.044457 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0cc9fe-db58-4100-bc16-809daa3e5dbd admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:06.071787 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1436/2871] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:46:06 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:06.075960 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcdb0fc-c499-4a14-8ce4-3e7a699539a0 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:06.088190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcdb0fc-c499-4a14-8ce4-3e7a699539a0 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:06.145933 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276485c-b663-4332-81de-a8e1e4cbc022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:06.146493 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276485c-b663-4332-81de-a8e1e4cbc022 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:06.146781 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276485c-b663-4332-81de-a8e1e4cbc022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:06.147058 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276485c-b663-4332-81de-a8e1e4cbc022 admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:06.147702 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1276485c-b663-4332-81de-a8e1e4cbc022 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:06.149632 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1276485c-b663-4332-81de-a8e1e4cbc022 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:06.171213 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1436/2872] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:46:06 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:06.175830 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0cc9fe-db58-4100-bc16-809daa3e5dbd admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:06.185986 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0cc9fe-db58-4100-bc16-809daa3e5dbd admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:06.191173 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de2e0b-659d-4d3d-9dfc-9324efbab14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:06.191771 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de2e0b-659d-4d3d-9dfc-9324efbab14f admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:06.192051 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de2e0b-659d-4d3d-9dfc-9324efbab14f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:06.192320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de2e0b-659d-4d3d-9dfc-9324efbab14f admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:06.192952 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de2e0b-659d-4d3d-9dfc-9324efbab14f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:06.194564 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de2e0b-659d-4d3d-9dfc-9324efbab14f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:06.204967 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1437/2873] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:06.209199 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276485c-b663-4332-81de-a8e1e4cbc022 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:06.219531 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276485c-b663-4332-81de-a8e1e4cbc022 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:06.224516 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc89e9-b426-4c02-934b-121459fb1217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:06.225125 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc89e9-b426-4c02-934b-121459fb1217 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:06.225411 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc89e9-b426-4c02-934b-121459fb1217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:06.225688 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc89e9-b426-4c02-934b-121459fb1217 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:06.226377 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dc89e9-b426-4c02-934b-121459fb1217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:06.228187 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dc89e9-b426-4c02-934b-121459fb1217 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:06.239933 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1437/2874] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:06.244217 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de2e0b-659d-4d3d-9dfc-9324efbab14f admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:06.254885 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de2e0b-659d-4d3d-9dfc-9324efbab14f admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:06.259450 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc704a39-5ef8-449b-b293-78dd515729ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:06.260121 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc704a39-5ef8-449b-b293-78dd515729ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:06.260398 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc704a39-5ef8-449b-b293-78dd515729ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:06.260670 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc704a39-5ef8-449b-b293-78dd515729ba admin admin] PATH_INFO: `/v3/projects/aecac0221ba54be8979d3e75c397a5c1/users/5c03adea3e404b2c9333247f0dbdc126/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:06.275183 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc704a39-5ef8-449b-b293-78dd515729ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=aecac0221ba54be8979d3e75c397a5c1, user_id=5c03adea3e404b2c9333247f0dbdc126, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:06.277848 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc704a39-5ef8-449b-b293-78dd515729ba admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:06.295403 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1438/2875] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:06 2022] PUT /identity/v3/projects/aecac0221ba54be8979d3e75c397a5c1/users/5c03adea3e404b2c9333247f0dbdc126/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:06.307388 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31c22a-7b3a-47fe-aec3-c12c80cfb685 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:06.307672 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31c22a-7b3a-47fe-aec3-c12c80cfb685 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:06.307945 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31c22a-7b3a-47fe-aec3-c12c80cfb685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:06.333423 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:06.336275 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-0c31c22a-7b3a-47fe-aec3-c12c80cfb685 None None] MFA Rules not processed for user `5c03adea3e404b2c9333247f0dbdc126`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:06.338317 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-0c31c22a-7b3a-47fe-aec3-c12c80cfb685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:06.403509 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1438/2876] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:46:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:06.407888 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc704a39-5ef8-449b-b293-78dd515729ba admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:06.468642 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc704a39-5ef8-449b-b293-78dd515729ba admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:06.474052 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e1ed0-a773-43c3-b415-55e2b04362b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:06.474686 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e1ed0-a773-43c3-b415-55e2b04362b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:06.474966 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e1ed0-a773-43c3-b415-55e2b04362b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:06.475237 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e1ed0-a773-43c3-b415-55e2b04362b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:06.475873 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231e1ed0-a773-43c3-b415-55e2b04362b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:06.477759 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231e1ed0-a773-43c3-b415-55e2b04362b8 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:06.506779 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1439/2877] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:46:06 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:06.510814 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31c22a-7b3a-47fe-aec3-c12c80cfb685 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:06.521931 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31c22a-7b3a-47fe-aec3-c12c80cfb685 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:06.580458 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7b6b0-e4f9-45a9-a981-6ba46a23bfef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:06.581070 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7b6b0-e4f9-45a9-a981-6ba46a23bfef admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:06.581350 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7b6b0-e4f9-45a9-a981-6ba46a23bfef admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:06.581621 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7b6b0-e4f9-45a9-a981-6ba46a23bfef admin admin] PATH_INFO: `/v3/users` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:06.582274 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d7b6b0-e4f9-45a9-a981-6ba46a23bfef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:06.584209 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d7b6b0-e4f9-45a9-a981-6ba46a23bfef admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:06.608894 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1439/2878] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:46:06 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:06.610165 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e1ed0-a773-43c3-b415-55e2b04362b8 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:06.621139 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e1ed0-a773-43c3-b415-55e2b04362b8 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:06.625777 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e0d845-caed-4d01-b2c6-696958a20436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:06.626420 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0d845-caed-4d01-b2c6-696958a20436 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:06.626703 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0d845-caed-4d01-b2c6-696958a20436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:06.626972 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0d845-caed-4d01-b2c6-696958a20436 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:06.627616 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e0d845-caed-4d01-b2c6-696958a20436 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:06.629192 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e0d845-caed-4d01-b2c6-696958a20436 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:06.640001 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1440/2879] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:46:06.644267 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7b6b0-e4f9-45a9-a981-6ba46a23bfef admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:06.654487 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7b6b0-e4f9-45a9-a981-6ba46a23bfef admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:06.659510 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a631b75-87e4-4bb6-8297-5a295772e8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:06.660161 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a631b75-87e4-4bb6-8297-5a295772e8a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:06.660445 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a631b75-87e4-4bb6-8297-5a295772e8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:06.660723 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a631b75-87e4-4bb6-8297-5a295772e8a5 admin admin] PATH_INFO: `/v3/projects/4cf99af92bfb4564a98686234c886003/users/f5458fb8531541338f40e4ed8283ff3e/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:06.676070 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a631b75-87e4-4bb6-8297-5a295772e8a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cf99af92bfb4564a98686234c886003, user_id=f5458fb8531541338f40e4ed8283ff3e, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:06.678778 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a631b75-87e4-4bb6-8297-5a295772e8a5 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:06.692097 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1440/2880] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:06 2022] PUT /identity/v3/projects/4cf99af92bfb4564a98686234c886003/users/f5458fb8531541338f40e4ed8283ff3e/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:06.696150 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e0d845-caed-4d01-b2c6-696958a20436 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:06.706344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e0d845-caed-4d01-b2c6-696958a20436 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:06.763057 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70efa909-d352-4e9b-982c-21cc2735939e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:06.763679 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efa909-d352-4e9b-982c-21cc2735939e admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:06.763961 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efa909-d352-4e9b-982c-21cc2735939e admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:06.764241 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efa909-d352-4e9b-982c-21cc2735939e admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:06.764879 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70efa909-d352-4e9b-982c-21cc2735939e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:06.766490 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70efa909-d352-4e9b-982c-21cc2735939e admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:06.772057 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:06.778093 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1441/2881] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:06.782128 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a631b75-87e4-4bb6-8297-5a295772e8a5 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:06.792901 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a631b75-87e4-4bb6-8297-5a295772e8a5 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:06.797448 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c122b-0ca8-4d39-9d38-0a5b65a30543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:06.798093 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c122b-0ca8-4d39-9d38-0a5b65a30543 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:06.798404 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c122b-0ca8-4d39-9d38-0a5b65a30543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:06.798678 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c122b-0ca8-4d39-9d38-0a5b65a30543 admin admin] PATH_INFO: `/v3/projects/4cf99af92bfb4564a98686234c886003/users/f5458fb8531541338f40e4ed8283ff3e/roles/e8ba063905484d22993bcf04830091a3` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:06.801320 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c122b-0ca8-4d39-9d38-0a5b65a30543 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cf99af92bfb4564a98686234c886003, user_id=f5458fb8531541338f40e4ed8283ff3e, role_id=e8ba063905484d22993bcf04830091a3)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:06.803975 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c122b-0ca8-4d39-9d38-0a5b65a30543 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:06.814187 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1441/2882] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:06 2022] PUT /identity/v3/projects/4cf99af92bfb4564a98686234c886003/users/f5458fb8531541338f40e4ed8283ff3e/roles/e8ba063905484d22993bcf04830091a3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:06.824791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b91c6-b217-4a9f-8641-8f095b4b6a58 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:06.825085 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b91c6-b217-4a9f-8641-8f095b4b6a58 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:06.825399 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b91c6-b217-4a9f-8641-8f095b4b6a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:06.854654 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-5e1b91c6-b217-4a9f-8641-8f095b4b6a58 None None] MFA Rules not processed for user `f5458fb8531541338f40e4ed8283ff3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:06.855867 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-5e1b91c6-b217-4a9f-8641-8f095b4b6a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:06.927736 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1442/2883] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:46:06 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:06.940183 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c122b-0ca8-4d39-9d38-0a5b65a30543 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:06.941978 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-522c122b-0ca8-4d39-9d38-0a5b65a30543 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:07.001929 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:07.024932 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c122b-0ca8-4d39-9d38-0a5b65a30543 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:07.028668 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1666ff01-c8a3-48de-a527-fe9ac2722468 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:07.029279 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666ff01-c8a3-48de-a527-fe9ac2722468 service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:07.029560 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666ff01-c8a3-48de-a527-fe9ac2722468 service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:07.029828 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666ff01-c8a3-48de-a527-fe9ac2722468 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:07.039897 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1666ff01-c8a3-48de-a527-fe9ac2722468 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:07.041341 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1666ff01-c8a3-48de-a527-fe9ac2722468 service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:07.051060 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1442/2884] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:06 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:07.356405 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b91c6-b217-4a9f-8641-8f095b4b6a58 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:07.366103 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b91c6-b217-4a9f-8641-8f095b4b6a58 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:07.370891 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52467b9c-39d0-40f3-b857-427417ed0230 tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5458fb8531541338f40e4ed8283ff3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cf99af92bfb4564a98686234c886003', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:07.371520 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52467b9c-39d0-40f3-b857-427417ed0230 tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:07.371802 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52467b9c-39d0-40f3-b857-427417ed0230 tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:07.372152 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52467b9c-39d0-40f3-b857-427417ed0230 tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:07.372812 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52467b9c-39d0-40f3-b857-427417ed0230 tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:46:07.373117 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52467b9c-39d0-40f3-b857-427417ed0230 tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:07.374775 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52467b9c-39d0-40f3-b857-427417ed0230 tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:07.387304 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1443/2885] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:46:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:46:07.391491 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1666ff01-c8a3-48de-a527-fe9ac2722468 service cinder] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:07.401153 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1666ff01-c8a3-48de-a527-fe9ac2722468 service cinder] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:07.405925 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c94e5b-5b9f-4597-ae15-089ca19b617a tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5458fb8531541338f40e4ed8283ff3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cf99af92bfb4564a98686234c886003', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:07.406567 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c94e5b-5b9f-4597-ae15-089ca19b617a tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:07.406855 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c94e5b-5b9f-4597-ae15-089ca19b617a tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:07.407130 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c94e5b-5b9f-4597-ae15-089ca19b617a tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:07.407783 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c94e5b-5b9f-4597-ae15-089ca19b617a tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:07.409718 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c94e5b-5b9f-4597-ae15-089ca19b617a tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:07.436191 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1443/2886] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:46:07 2022] POST /identity/v3/projects => generated 353 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:07.440769 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52467b9c-39d0-40f3-b857-427417ed0230 tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:07.451033 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52467b9c-39d0-40f3-b857-427417ed0230 tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:07.507487 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b997abd-2ba5-4bd0-ac3f-2e553cd02726 tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5458fb8531541338f40e4ed8283ff3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cf99af92bfb4564a98686234c886003', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:07.508107 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b997abd-2ba5-4bd0-ac3f-2e553cd02726 tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:07.508400 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b997abd-2ba5-4bd0-ac3f-2e553cd02726 tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:07.508705 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b997abd-2ba5-4bd0-ac3f-2e553cd02726 tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:07.509360 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b997abd-2ba5-4bd0-ac3f-2e553cd02726 tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:46:07.509664 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b997abd-2ba5-4bd0-ac3f-2e553cd02726 tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:07.511339 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b997abd-2ba5-4bd0-ac3f-2e553cd02726 tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:07.524268 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1444/2887] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:46:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:46:07.560448 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c94e5b-5b9f-4597-ae15-089ca19b617a tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:07.571679 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c94e5b-5b9f-4597-ae15-089ca19b617a tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:07.576323 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2def21-b360-45c1-bd02-bfb60e43eda8 tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5458fb8531541338f40e4ed8283ff3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cf99af92bfb4564a98686234c886003', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:07.576961 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2def21-b360-45c1-bd02-bfb60e43eda8 tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:07.577247 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2def21-b360-45c1-bd02-bfb60e43eda8 tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:07.577524 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2def21-b360-45c1-bd02-bfb60e43eda8 tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] PATH_INFO: `/v3/projects/e9ffae9c05014f499073b01286a49443` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:07.578693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2def21-b360-45c1-bd02-bfb60e43eda8 tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e9ffae9c05014f499073b01286a49443)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:07.580655 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2def21-b360-45c1-bd02-bfb60e43eda8 tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:07.628426 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-ee2def21-b360-45c1-bd02-bfb60e43eda8 tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9ffae9c05014f499073b01286a49443', 'request_id': 'req-ee2def21-b360-45c1-bd02-bfb60e43eda8'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:07.636029 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-ee2def21-b360-45c1-bd02-bfb60e43eda8 tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9ffae9c05014f499073b01286a49443', 'request_id': 'req-ee2def21-b360-45c1-bd02-bfb60e43eda8'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:07.643201 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-ee2def21-b360-45c1-bd02-bfb60e43eda8 tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9ffae9c05014f499073b01286a49443', 'request_id': 'req-ee2def21-b360-45c1-bd02-bfb60e43eda8'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:07.653539 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-ee2def21-b360-45c1-bd02-bfb60e43eda8 tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] The token cache is being invalidate because project e9ffae9c05014f499073b01286a49443 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:07.653830 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-ee2def21-b360-45c1-bd02-bfb60e43eda8 tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:07.654531 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-ee2def21-b360-45c1-bd02-bfb60e43eda8 tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:07.656016 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1444/2888] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:07 2022] DELETE /identity/v3/projects/e9ffae9c05014f499073b01286a49443 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:07.844822 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b997abd-2ba5-4bd0-ac3f-2e553cd02726 tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:07.846560 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-1b997abd-2ba5-4bd0-ac3f-2e553cd02726 tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:07.910897 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b997abd-2ba5-4bd0-ac3f-2e553cd02726 tempest-VolumeQuotaClassesTest-2071865775 tempest-VolumeQuotaClassesTest-2071865775-project-admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:07.919941 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=100 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:07.966820 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a54c36-987b-4008-8ad1-a16e575801c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:07.967456 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a54c36-987b-4008-8ad1-a16e575801c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:07.967738 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a54c36-987b-4008-8ad1-a16e575801c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:07.968013 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a54c36-987b-4008-8ad1-a16e575801c1 admin admin] PATH_INFO: `/v3/users/5c03adea3e404b2c9333247f0dbdc126` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:07.969177 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a54c36-987b-4008-8ad1-a16e575801c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c03adea3e404b2c9333247f0dbdc126)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:07.971610 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a54c36-987b-4008-8ad1-a16e575801c1 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:08.019471 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-74a54c36-987b-4008-8ad1-a16e575801c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c03adea3e404b2c9333247f0dbdc126'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:08.027473 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-74a54c36-987b-4008-8ad1-a16e575801c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c03adea3e404b2c9333247f0dbdc126'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:08.028204 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-74a54c36-987b-4008-8ad1-a16e575801c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c03adea3e404b2c9333247f0dbdc126'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:08.040741 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-74a54c36-987b-4008-8ad1-a16e575801c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c03adea3e404b2c9333247f0dbdc126'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:08.041480 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-74a54c36-987b-4008-8ad1-a16e575801c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c03adea3e404b2c9333247f0dbdc126'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:08.054131 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1445/2889] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:46:07 2022] DELETE /identity/v3/users/5c03adea3e404b2c9333247f0dbdc126 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:08.058740 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e25ca-9d74-473a-a84d-63cbd8b8784a None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:08.059240 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e25ca-9d74-473a-a84d-63cbd8b8784a None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:08.059363 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e25ca-9d74-473a-a84d-63cbd8b8784a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:08.078659 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-b13e25ca-9d74-473a-a84d-63cbd8b8784a None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:08.079849 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-b13e25ca-9d74-473a-a84d-63cbd8b8784a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:08.140062 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1445/2890] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:46:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:08.149905 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a54c36-987b-4008-8ad1-a16e575801c1 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:08.151344 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-74a54c36-987b-4008-8ad1-a16e575801c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:08.209129 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a54c36-987b-4008-8ad1-a16e575801c1 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:08.212786 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe3bad-29d1-4e1c-a924-f6afdabee2ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:08.213440 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe3bad-29d1-4e1c-a924-f6afdabee2ce service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:08.213721 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe3bad-29d1-4e1c-a924-f6afdabee2ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:08.213991 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe3bad-29d1-4e1c-a924-f6afdabee2ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:08.223728 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe3bad-29d1-4e1c-a924-f6afdabee2ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:08.225144 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe3bad-29d1-4e1c-a924-f6afdabee2ce service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:08.234552 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1446/2891] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:08.511866 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e25ca-9d74-473a-a84d-63cbd8b8784a None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:08.514457 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-b13e25ca-9d74-473a-a84d-63cbd8b8784a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:08.527934 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e25ca-9d74-473a-a84d-63cbd8b8784a None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:08.532601 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5aaaf-12c7-4266-8edb-e4bb5e9d1ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:08.533281 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5aaaf-12c7-4266-8edb-e4bb5e9d1ca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:08.533566 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5aaaf-12c7-4266-8edb-e4bb5e9d1ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:08.534635 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5aaaf-12c7-4266-8edb-e4bb5e9d1ca4 admin admin] PATH_INFO: `/v3/projects/aecac0221ba54be8979d3e75c397a5c1` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:08.536074 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff5aaaf-12c7-4266-8edb-e4bb5e9d1ca4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aecac0221ba54be8979d3e75c397a5c1)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:08.538047 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff5aaaf-12c7-4266-8edb-e4bb5e9d1ca4 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:08.576236 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:08.588800 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-9ff5aaaf-12c7-4266-8edb-e4bb5e9d1ca4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aecac0221ba54be8979d3e75c397a5c1', 'request_id': 'req-9ff5aaaf-12c7-4266-8edb-e4bb5e9d1ca4'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:08.596973 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-9ff5aaaf-12c7-4266-8edb-e4bb5e9d1ca4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aecac0221ba54be8979d3e75c397a5c1', 'request_id': 'req-9ff5aaaf-12c7-4266-8edb-e4bb5e9d1ca4'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:08.604048 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-9ff5aaaf-12c7-4266-8edb-e4bb5e9d1ca4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aecac0221ba54be8979d3e75c397a5c1', 'request_id': 'req-9ff5aaaf-12c7-4266-8edb-e4bb5e9d1ca4'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:08.614281 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-9ff5aaaf-12c7-4266-8edb-e4bb5e9d1ca4 admin admin] The token cache is being invalidate because project aecac0221ba54be8979d3e75c397a5c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:08.614573 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-9ff5aaaf-12c7-4266-8edb-e4bb5e9d1ca4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:08.615209 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-9ff5aaaf-12c7-4266-8edb-e4bb5e9d1ca4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:08.616694 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1446/2892] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:08 2022] DELETE /identity/v3/projects/aecac0221ba54be8979d3e75c397a5c1 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:08.620815 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe3bad-29d1-4e1c-a924-f6afdabee2ce service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:08.630548 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe3bad-29d1-4e1c-a924-f6afdabee2ce service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:08.663588 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:08.688714 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be015a-1a09-4a01-b57a-41c65a82986e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:08.689346 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be015a-1a09-4a01-b57a-41c65a82986e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:08.689629 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be015a-1a09-4a01-b57a-41c65a82986e admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:08.690458 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be015a-1a09-4a01-b57a-41c65a82986e admin admin] PATH_INFO: `/v3/users/f5458fb8531541338f40e4ed8283ff3e` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:08.691584 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be015a-1a09-4a01-b57a-41c65a82986e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5458fb8531541338f40e4ed8283ff3e)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:08.693542 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be015a-1a09-4a01-b57a-41c65a82986e admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:08.739675 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-06be015a-1a09-4a01-b57a-41c65a82986e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5458fb8531541338f40e4ed8283ff3e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:08.747498 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-06be015a-1a09-4a01-b57a-41c65a82986e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5458fb8531541338f40e4ed8283ff3e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:08.748207 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-06be015a-1a09-4a01-b57a-41c65a82986e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5458fb8531541338f40e4ed8283ff3e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:08.761031 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-06be015a-1a09-4a01-b57a-41c65a82986e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5458fb8531541338f40e4ed8283ff3e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:08.761882 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-06be015a-1a09-4a01-b57a-41c65a82986e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5458fb8531541338f40e4ed8283ff3e'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:08.774875 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1447/2893] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:46:08 2022] DELETE /identity/v3/users/f5458fb8531541338f40e4ed8283ff3e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:09.061851 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5aaaf-12c7-4266-8edb-e4bb5e9d1ca4 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:09.063369 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-9ff5aaaf-12c7-4266-8edb-e4bb5e9d1ca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:09.078557 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5aaaf-12c7-4266-8edb-e4bb5e9d1ca4 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:09.138298 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fba869-80d4-432c-89be-2f588a38c3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:09.138861 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fba869-80d4-432c-89be-2f588a38c3ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:09.139144 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fba869-80d4-432c-89be-2f588a38c3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:09.139419 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fba869-80d4-432c-89be-2f588a38c3ee admin admin] PATH_INFO: `/v3/projects/4cf99af92bfb4564a98686234c886003` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:09.140607 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fba869-80d4-432c-89be-2f588a38c3ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cf99af92bfb4564a98686234c886003)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:09.142559 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fba869-80d4-432c-89be-2f588a38c3ee admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:09.188668 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-96fba869-80d4-432c-89be-2f588a38c3ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cf99af92bfb4564a98686234c886003', 'request_id': 'req-96fba869-80d4-432c-89be-2f588a38c3ee'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:09.196737 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-96fba869-80d4-432c-89be-2f588a38c3ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cf99af92bfb4564a98686234c886003', 'request_id': 'req-96fba869-80d4-432c-89be-2f588a38c3ee'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:09.203782 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-96fba869-80d4-432c-89be-2f588a38c3ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cf99af92bfb4564a98686234c886003', 'request_id': 'req-96fba869-80d4-432c-89be-2f588a38c3ee'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:09.214397 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-96fba869-80d4-432c-89be-2f588a38c3ee admin admin] The token cache is being invalidate because project 4cf99af92bfb4564a98686234c886003 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:09.214693 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-96fba869-80d4-432c-89be-2f588a38c3ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:09.215360 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-96fba869-80d4-432c-89be-2f588a38c3ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:09.216837 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1447/2894] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:09 2022] DELETE /identity/v3/projects/4cf99af92bfb4564a98686234c886003 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:09.223178 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978fe61-3ce5-467e-bbf5-95bc60360c00 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:09.223692 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978fe61-3ce5-467e-bbf5-95bc60360c00 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:09.223816 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978fe61-3ce5-467e-bbf5-95bc60360c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:09.243594 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-a978fe61-3ce5-467e-bbf5-95bc60360c00 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:09.244797 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-a978fe61-3ce5-467e-bbf5-95bc60360c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:09.303976 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1448/2895] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:46:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:09.312034 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032fca7-da24-411b-bd95-4ca1a545d91c None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:09.312320 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032fca7-da24-411b-bd95-4ca1a545d91c None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:09.312598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032fca7-da24-411b-bd95-4ca1a545d91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:09.341924 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-e032fca7-da24-411b-bd95-4ca1a545d91c None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:09.343159 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-e032fca7-da24-411b-bd95-4ca1a545d91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:09.351350 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1448/2896] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:46:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:09.355656 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a978fe61-3ce5-467e-bbf5-95bc60360c00 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:09.365909 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a978fe61-3ce5-467e-bbf5-95bc60360c00 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:09.372437 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751959e0-b7da-4212-8b15-7eb6302d5a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:09.373071 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751959e0-b7da-4212-8b15-7eb6302d5a6d admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:09.373354 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751959e0-b7da-4212-8b15-7eb6302d5a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:09.373629 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751959e0-b7da-4212-8b15-7eb6302d5a6d admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:09.374303 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751959e0-b7da-4212-8b15-7eb6302d5a6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:46:09.374612 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751959e0-b7da-4212-8b15-7eb6302d5a6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:09.376193 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751959e0-b7da-4212-8b15-7eb6302d5a6d admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:09.389037 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1449/2897] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:46:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:46:09.393298 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e032fca7-da24-411b-bd95-4ca1a545d91c None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:09.403532 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e032fca7-da24-411b-bd95-4ca1a545d91c None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:09.408603 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e8c43-c6f7-4d50-8d8a-8774a5950919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:09.409214 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e8c43-c6f7-4d50-8d8a-8774a5950919 admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:09.409493 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e8c43-c6f7-4d50-8d8a-8774a5950919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:09.409798 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e8c43-c6f7-4d50-8d8a-8774a5950919 admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:09.410480 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e8c43-c6f7-4d50-8d8a-8774a5950919 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:09.412421 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e8c43-c6f7-4d50-8d8a-8774a5950919 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:09.440660 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1449/2898] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:46:09 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:09.444924 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751959e0-b7da-4212-8b15-7eb6302d5a6d admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:09.455441 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751959e0-b7da-4212-8b15-7eb6302d5a6d admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:09.496248 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:09.515025 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f75420-4641-4fe2-b27f-2f1e92c33067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:09.515681 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f75420-4641-4fe2-b27f-2f1e92c33067 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:09.515951 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f75420-4641-4fe2-b27f-2f1e92c33067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:09.516223 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f75420-4641-4fe2-b27f-2f1e92c33067 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:09.516866 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f75420-4641-4fe2-b27f-2f1e92c33067 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:09.518868 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f75420-4641-4fe2-b27f-2f1e92c33067 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:09.540975 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1450/2899] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:46:09 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:09.545323 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e8c43-c6f7-4d50-8d8a-8774a5950919 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:09.557079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e8c43-c6f7-4d50-8d8a-8774a5950919 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:09.561685 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db63f3-550d-4b88-b6fe-28a3922dbf08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:09.562336 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db63f3-550d-4b88-b6fe-28a3922dbf08 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:09.562625 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db63f3-550d-4b88-b6fe-28a3922dbf08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:09.562895 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db63f3-550d-4b88-b6fe-28a3922dbf08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:09.563569 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2db63f3-550d-4b88-b6fe-28a3922dbf08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:09.565201 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2db63f3-550d-4b88-b6fe-28a3922dbf08 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:09.575320 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1450/2900] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:09.579595 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f75420-4641-4fe2-b27f-2f1e92c33067 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:09.589935 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f75420-4641-4fe2-b27f-2f1e92c33067 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:09.594646 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f2758-6bc8-4d46-b992-117a5cab249a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:09.595293 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f2758-6bc8-4d46-b992-117a5cab249a admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:09.595580 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f2758-6bc8-4d46-b992-117a5cab249a admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:09.595851 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f2758-6bc8-4d46-b992-117a5cab249a admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:09.597016 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f2758-6bc8-4d46-b992-117a5cab249a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:09.598882 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f2758-6bc8-4d46-b992-117a5cab249a admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:09.609747 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1451/2901] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:09.614839 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db63f3-550d-4b88-b6fe-28a3922dbf08 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:09.624806 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db63f3-550d-4b88-b6fe-28a3922dbf08 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:09.629437 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f6918-7e44-44db-b638-4d7cff477779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:09.630084 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f6918-7e44-44db-b638-4d7cff477779 admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:09.630400 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f6918-7e44-44db-b638-4d7cff477779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:09.630675 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f6918-7e44-44db-b638-4d7cff477779 admin admin] PATH_INFO: `/v3/projects/b0b66ffb312e4973b8d2f94494679327/users/9af8c4cf99ee4953a6c31a10d642a3e8/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:09.641702 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:09.646402 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52f6918-7e44-44db-b638-4d7cff477779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0b66ffb312e4973b8d2f94494679327, user_id=9af8c4cf99ee4953a6c31a10d642a3e8, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:09.649240 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52f6918-7e44-44db-b638-4d7cff477779 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:09.669857 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1451/2902] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:09 2022] PUT /identity/v3/projects/b0b66ffb312e4973b8d2f94494679327/users/9af8c4cf99ee4953a6c31a10d642a3e8/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:09.736559 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b418a5-39a8-4cd9-8537-d34c60fadfa2 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:09.736936 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b418a5-39a8-4cd9-8537-d34c60fadfa2 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:09.737715 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b418a5-39a8-4cd9-8537-d34c60fadfa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:09.765354 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-21b418a5-39a8-4cd9-8537-d34c60fadfa2 None None] MFA Rules not processed for user `9af8c4cf99ee4953a6c31a10d642a3e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:09.766629 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-21b418a5-39a8-4cd9-8537-d34c60fadfa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:09.830340 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1452/2903] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:46:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:09.841853 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f6918-7e44-44db-b638-4d7cff477779 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:09.843553 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-f52f6918-7e44-44db-b638-4d7cff477779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:09.900909 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f6918-7e44-44db-b638-4d7cff477779 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:09.904462 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6baabfc-7f29-4a04-9c12-c58680642b39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:09.905067 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6baabfc-7f29-4a04-9c12-c58680642b39 service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:09.905347 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6baabfc-7f29-4a04-9c12-c58680642b39 service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:09.905626 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6baabfc-7f29-4a04-9c12-c58680642b39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:09.915086 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6baabfc-7f29-4a04-9c12-c58680642b39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:09.916540 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6baabfc-7f29-4a04-9c12-c58680642b39 service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:09.925946 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1452/2904] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:09.947028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b418a5-39a8-4cd9-8537-d34c60fadfa2 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:10.010021 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b418a5-39a8-4cd9-8537-d34c60fadfa2 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:10.014649 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5eb45-9eb4-42ff-8553-c170be276081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:10.015285 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5eb45-9eb4-42ff-8553-c170be276081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:10.015568 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5eb45-9eb4-42ff-8553-c170be276081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:10.015845 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5eb45-9eb4-42ff-8553-c170be276081 admin admin] PATH_INFO: `/v3/users/9af8c4cf99ee4953a6c31a10d642a3e8` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:10.016997 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5eb45-9eb4-42ff-8553-c170be276081 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9af8c4cf99ee4953a6c31a10d642a3e8)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:10.019002 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5eb45-9eb4-42ff-8553-c170be276081 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:10.064820 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-16a5eb45-9eb4-42ff-8553-c170be276081 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9af8c4cf99ee4953a6c31a10d642a3e8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:10.069254 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:10.073917 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-16a5eb45-9eb4-42ff-8553-c170be276081 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9af8c4cf99ee4953a6c31a10d642a3e8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:10.075310 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-16a5eb45-9eb4-42ff-8553-c170be276081 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9af8c4cf99ee4953a6c31a10d642a3e8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:10.089154 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-16a5eb45-9eb4-42ff-8553-c170be276081 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9af8c4cf99ee4953a6c31a10d642a3e8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:10.089830 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-16a5eb45-9eb4-42ff-8553-c170be276081 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9af8c4cf99ee4953a6c31a10d642a3e8'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:10.102097 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1453/2905] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:46:09 2022] DELETE /identity/v3/users/9af8c4cf99ee4953a6c31a10d642a3e8 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:10.108014 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b89e17-7280-4201-9c62-9d50cff15121 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:10.108588 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b89e17-7280-4201-9c62-9d50cff15121 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:10.108721 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b89e17-7280-4201-9c62-9d50cff15121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:10.129934 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-63b89e17-7280-4201-9c62-9d50cff15121 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:10.131190 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-63b89e17-7280-4201-9c62-9d50cff15121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:10.188609 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1453/2906] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:46:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:10.198991 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5eb45-9eb4-42ff-8553-c170be276081 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:10.200412 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-16a5eb45-9eb4-42ff-8553-c170be276081 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:10.259529 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5eb45-9eb4-42ff-8553-c170be276081 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:10.266898 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723812ff-cfb5-4f75-8634-d429e248dd37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:10.267515 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723812ff-cfb5-4f75-8634-d429e248dd37 service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:10.267791 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723812ff-cfb5-4f75-8634-d429e248dd37 service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:10.268061 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723812ff-cfb5-4f75-8634-d429e248dd37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:10.277543 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723812ff-cfb5-4f75-8634-d429e248dd37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:10.278997 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723812ff-cfb5-4f75-8634-d429e248dd37 service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:10.288848 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1454/2907] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:10.560662 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b89e17-7280-4201-9c62-9d50cff15121 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:10.562104 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-63b89e17-7280-4201-9c62-9d50cff15121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:10.575079 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b89e17-7280-4201-9c62-9d50cff15121 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:10.579907 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a3525a-3d0e-4cb1-9ce0-575c0ceef397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:10.581814 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3525a-3d0e-4cb1-9ce0-575c0ceef397 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:10.582175 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3525a-3d0e-4cb1-9ce0-575c0ceef397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:10.582504 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3525a-3d0e-4cb1-9ce0-575c0ceef397 admin admin] PATH_INFO: `/v3/projects/b0b66ffb312e4973b8d2f94494679327` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:10.583798 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a3525a-3d0e-4cb1-9ce0-575c0ceef397 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0b66ffb312e4973b8d2f94494679327)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:10.585714 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a3525a-3d0e-4cb1-9ce0-575c0ceef397 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:10.632241 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-56a3525a-3d0e-4cb1-9ce0-575c0ceef397 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0b66ffb312e4973b8d2f94494679327', 'request_id': 'req-56a3525a-3d0e-4cb1-9ce0-575c0ceef397'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:10.639501 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-56a3525a-3d0e-4cb1-9ce0-575c0ceef397 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0b66ffb312e4973b8d2f94494679327', 'request_id': 'req-56a3525a-3d0e-4cb1-9ce0-575c0ceef397'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:10.646571 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-56a3525a-3d0e-4cb1-9ce0-575c0ceef397 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0b66ffb312e4973b8d2f94494679327', 'request_id': 'req-56a3525a-3d0e-4cb1-9ce0-575c0ceef397'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:10.656817 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-56a3525a-3d0e-4cb1-9ce0-575c0ceef397 admin admin] The token cache is being invalidate because project b0b66ffb312e4973b8d2f94494679327 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:10.657107 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-56a3525a-3d0e-4cb1-9ce0-575c0ceef397 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:10.657752 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-56a3525a-3d0e-4cb1-9ce0-575c0ceef397 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:10.659817 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1454/2908] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:10 2022] DELETE /identity/v3/projects/b0b66ffb312e4973b8d2f94494679327 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:10.665440 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89070b-cc4d-4eb6-af11-718a6a7be7c7 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:10.665729 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89070b-cc4d-4eb6-af11-718a6a7be7c7 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:10.666001 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89070b-cc4d-4eb6-af11-718a6a7be7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:10.685163 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-ff89070b-cc4d-4eb6-af11-718a6a7be7c7 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:10.686406 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ff89070b-cc4d-4eb6-af11-718a6a7be7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:10.745140 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1455/2909] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:46:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:10.752728 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f51e77-c110-4199-9e1c-d5dc97672db5 None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:10.753019 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f51e77-c110-4199-9e1c-d5dc97672db5 None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:10.753299 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f51e77-c110-4199-9e1c-d5dc97672db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:10.773338 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-c7f51e77-c110-4199-9e1c-d5dc97672db5 None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:10.774590 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-c7f51e77-c110-4199-9e1c-d5dc97672db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:10.782598 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1455/2910] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:46:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:10.787396 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89070b-cc4d-4eb6-af11-718a6a7be7c7 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:10.804574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89070b-cc4d-4eb6-af11-718a6a7be7c7 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:10.809308 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef571943-8d51-4885-a794-b7e52213815f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:10.810039 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef571943-8d51-4885-a794-b7e52213815f admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:10.810351 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef571943-8d51-4885-a794-b7e52213815f admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:10.810626 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef571943-8d51-4885-a794-b7e52213815f admin admin] PATH_INFO: `/v3/domains` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:10.811272 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef571943-8d51-4885-a794-b7e52213815f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81551) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:46:10.811574 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef571943-8d51-4885-a794-b7e52213815f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:10.813162 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef571943-8d51-4885-a794-b7e52213815f admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:10.825158 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1456/2911] 10.222.0.18 () {58 vars in 1153 bytes} [Thu Sep 1 16:46:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:46:10.829520 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f51e77-c110-4199-9e1c-d5dc97672db5 None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:10.839432 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f51e77-c110-4199-9e1c-d5dc97672db5 None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:10.844292 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7648e0-92d5-4199-a950-caa797c0b08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:10.844882 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7648e0-92d5-4199-a950-caa797c0b08c admin admin] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:10.845161 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7648e0-92d5-4199-a950-caa797c0b08c admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:10.845432 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7648e0-92d5-4199-a950-caa797c0b08c admin admin] PATH_INFO: `/v3/projects` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:10.846055 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7648e0-92d5-4199-a950-caa797c0b08c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:10.848937 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7648e0-92d5-4199-a950-caa797c0b08c admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:10.854484 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:10.881010 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1456/2912] 10.222.0.18 () {60 vars in 1153 bytes} [Thu Sep 1 16:46:10 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:10.885041 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef571943-8d51-4885-a794-b7e52213815f admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:10.895704 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef571943-8d51-4885-a794-b7e52213815f admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:10.951737 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56d5f5-d33b-410c-ad07-4fc3c861bbf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:10.952351 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56d5f5-d33b-410c-ad07-4fc3c861bbf1 admin admin] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:10.952634 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56d5f5-d33b-410c-ad07-4fc3c861bbf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:10.952908 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56d5f5-d33b-410c-ad07-4fc3c861bbf1 admin admin] PATH_INFO: `/v3/users` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:10.953544 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db56d5f5-d33b-410c-ad07-4fc3c861bbf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:10.955488 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db56d5f5-d33b-410c-ad07-4fc3c861bbf1 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:10.964028 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:10.978973 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1457/2913] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Sep 1 16:46:10 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:10.983155 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7648e0-92d5-4199-a950-caa797c0b08c admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:10.993482 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7648e0-92d5-4199-a950-caa797c0b08c admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:10.998068 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4bd01b-f580-4376-84cf-b8a587189dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:10.998629 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4bd01b-f580-4376-84cf-b8a587189dd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:10.998909 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4bd01b-f580-4376-84cf-b8a587189dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:10.999178 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4bd01b-f580-4376-84cf-b8a587189dd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:10.999820 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4bd01b-f580-4376-84cf-b8a587189dd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:11.001431 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4bd01b-f580-4376-84cf-b8a587189dd3 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:11.012595 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1457/2914] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:11.016614 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56d5f5-d33b-410c-ad07-4fc3c861bbf1 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:11.027542 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56d5f5-d33b-410c-ad07-4fc3c861bbf1 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:11.032221 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cce7b0-9b37-4e0b-b926-bb4842487762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:11.032811 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cce7b0-9b37-4e0b-b926-bb4842487762 admin admin] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:11.033095 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cce7b0-9b37-4e0b-b926-bb4842487762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:11.033369 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cce7b0-9b37-4e0b-b926-bb4842487762 admin admin] PATH_INFO: `/v3/roles` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:11.034012 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cce7b0-9b37-4e0b-b926-bb4842487762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:11.035637 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cce7b0-9b37-4e0b-b926-bb4842487762 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:11.046422 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1458/2915] 10.222.0.18 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:11.050685 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4bd01b-f580-4376-84cf-b8a587189dd3 admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:11.060903 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4bd01b-f580-4376-84cf-b8a587189dd3 admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:11.066319 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334cfbf-1586-414a-bd69-b879017cb93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:11.066930 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334cfbf-1586-414a-bd69-b879017cb93d admin admin] REQUEST_METHOD: `PUT` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:11.067214 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334cfbf-1586-414a-bd69-b879017cb93d admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:11.067496 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334cfbf-1586-414a-bd69-b879017cb93d admin admin] PATH_INFO: `/v3/projects/50b4838c797b4062bb2419d959ed9ecb/users/dff7135b68f649d79b2d152777da557c/roles/45545334e1fc473c9d85fc91a9b0a35c` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:11.084331 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1334cfbf-1586-414a-bd69-b879017cb93d admin admin] RBAC: Authorizing `identity:create_grant(project_id=50b4838c797b4062bb2419d959ed9ecb, user_id=dff7135b68f649d79b2d152777da557c, role_id=45545334e1fc473c9d85fc91a9b0a35c)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:11.087046 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1334cfbf-1586-414a-bd69-b879017cb93d admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:11.101378 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1458/2916] 10.222.0.18 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:11 2022] PUT /identity/v3/projects/50b4838c797b4062bb2419d959ed9ecb/users/dff7135b68f649d79b2d152777da557c/roles/45545334e1fc473c9d85fc91a9b0a35c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:11.111849 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7630b7-999a-4a30-a1fb-f034cf525fe9 None None] REQUEST_METHOD: `POST` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:11.112142 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7630b7-999a-4a30-a1fb-f034cf525fe9 None None] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:11.112485 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7630b7-999a-4a30-a1fb-f034cf525fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:11.141346 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.auth.core [None req-ba7630b7-999a-4a30-a1fb-f034cf525fe9 None None] MFA Rules not processed for user `dff7135b68f649d79b2d152777da557c`. Rule list: `[]` (Enabled: `True`). {{(pid=81551) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:11.142610 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-ba7630b7-999a-4a30-a1fb-f034cf525fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:11.206465 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1459/2917] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:46:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:11.217718 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334cfbf-1586-414a-bd69-b879017cb93d admin admin] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:11.219448 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-1334cfbf-1586-414a-bd69-b879017cb93d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:11.277804 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334cfbf-1586-414a-bd69-b879017cb93d admin admin] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:11.281368 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c5ed1-5e2f-4b40-ae66-bd180886cb8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b78747bb9140457896fe62d4f537592c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:11.281995 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c5ed1-5e2f-4b40-ae66-bd180886cb8b service cinder] REQUEST_METHOD: `GET` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:11.282295 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c5ed1-5e2f-4b40-ae66-bd180886cb8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:11.282575 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c5ed1-5e2f-4b40-ae66-bd180886cb8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:11.290984 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c5ed1-5e2f-4b40-ae66-bd180886cb8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:11.292428 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c5ed1-5e2f-4b40-ae66-bd180886cb8b service cinder] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:11.301327 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1459/2918] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:21.202808 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:INSERT=3,keystone:SELECT=39 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:21.297271 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=46,keystone:INSERT=2 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:45.283836 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7630b7-999a-4a30-a1fb-f034cf525fe9 None None] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:45.345319 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7630b7-999a-4a30-a1fb-f034cf525fe9 None None] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:45.350444 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea72c4-7686-4917-a038-db0d05db8735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:45.351092 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea72c4-7686-4917-a038-db0d05db8735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:45.351384 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea72c4-7686-4917-a038-db0d05db8735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:45.351669 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea72c4-7686-4917-a038-db0d05db8735 admin admin] PATH_INFO: `/v3/users/dff7135b68f649d79b2d152777da557c` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:45.352821 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea72c4-7686-4917-a038-db0d05db8735 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dff7135b68f649d79b2d152777da557c)` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:45.354828 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea72c4-7686-4917-a038-db0d05db8735 admin admin] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:45.401930 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0bea72c4-7686-4917-a038-db0d05db8735 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dff7135b68f649d79b2d152777da557c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:45.409478 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0bea72c4-7686-4917-a038-db0d05db8735 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dff7135b68f649d79b2d152777da557c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:45.410115 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0bea72c4-7686-4917-a038-db0d05db8735 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dff7135b68f649d79b2d152777da557c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:45.422218 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0bea72c4-7686-4917-a038-db0d05db8735 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dff7135b68f649d79b2d152777da557c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:45.422895 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.notifications [None req-0bea72c4-7686-4917-a038-db0d05db8735 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dff7135b68f649d79b2d152777da557c'} {{(pid=81551) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:45.435332 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1460/2919] 10.222.0.18 () {58 vars in 1224 bytes} [Thu Sep 1 16:46:45 2022] DELETE /identity/v3/users/dff7135b68f649d79b2d152777da557c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:45.439832 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff637a3-ec06-46d3-b27b-564a15ad5d3b None None] REQUEST_METHOD: `POST` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:45.440879 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff637a3-ec06-46d3-b27b-564a15ad5d3b None None] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:45.441041 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff637a3-ec06-46d3-b27b-564a15ad5d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:45.460027 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.auth.core [None req-8ff637a3-ec06-46d3-b27b-564a15ad5d3b None None] MFA Rules not processed for user `eed887c1c93d4f24959b0f90e215c424`. Rule list: `[]` (Enabled: `True`). {{(pid=81552) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:45.461230 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-8ff637a3-ec06-46d3-b27b-564a15ad5d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:45.521352 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1460/2920] 10.222.0.18 () {58 vars in 958 bytes} [Thu Sep 1 16:46:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:45.531117 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea72c4-7686-4917-a038-db0d05db8735 admin admin] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:45.532569 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0bea72c4-7686-4917-a038-db0d05db8735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:45.590407 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea72c4-7686-4917-a038-db0d05db8735 admin admin] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:45.594115 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d77ad9-f26f-4461-9178-947cd7b0855f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffa0f2160a934906b59e6f845edb0776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:45.594828 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d77ad9-f26f-4461-9178-947cd7b0855f service neutron] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:45.595117 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d77ad9-f26f-4461-9178-947cd7b0855f service neutron] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:45.595390 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d77ad9-f26f-4461-9178-947cd7b0855f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:45.605169 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d77ad9-f26f-4461-9178-947cd7b0855f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:45.606654 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d77ad9-f26f-4461-9178-947cd7b0855f service neutron] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:45.616615 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1461/2921] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:45.884335 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff637a3-ec06-46d3-b27b-564a15ad5d3b None None] Authenticating user token {{(pid=81552) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:45.885777 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.fernet_utils [None req-8ff637a3-ec06-46d3-b27b-564a15ad5d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81552) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:45.898558 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff637a3-ec06-46d3-b27b-564a15ad5d3b None None] Validating token access rules against request {{(pid=81552) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:45.903631 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b7ce93-145f-4270-bcb3-fd875fd36845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eed887c1c93d4f24959b0f90e215c424', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d95c560fa484406b7dcc65daad0e99e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81552) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:45.904271 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7ce93-145f-4270-bcb3-fd875fd36845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:45.904554 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7ce93-145f-4270-bcb3-fd875fd36845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:45.904829 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7ce93-145f-4270-bcb3-fd875fd36845 admin admin] PATH_INFO: `/v3/projects/50b4838c797b4062bb2419d959ed9ecb` {{(pid=81552) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:45.905966 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b7ce93-145f-4270-bcb3-fd875fd36845 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50b4838c797b4062bb2419d959ed9ecb)` {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:45.907942 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b7ce93-145f-4270-bcb3-fd875fd36845 admin admin] RBAC: Authorization granted {{(pid=81552) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:45.954738 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-96b7ce93-145f-4270-bcb3-fd875fd36845 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50b4838c797b4062bb2419d959ed9ecb', 'request_id': 'req-96b7ce93-145f-4270-bcb3-fd875fd36845'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:45.962191 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-96b7ce93-145f-4270-bcb3-fd875fd36845 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50b4838c797b4062bb2419d959ed9ecb', 'request_id': 'req-96b7ce93-145f-4270-bcb3-fd875fd36845'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:45.968397 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-96b7ce93-145f-4270-bcb3-fd875fd36845 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50b4838c797b4062bb2419d959ed9ecb', 'request_id': 'req-96b7ce93-145f-4270-bcb3-fd875fd36845'} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:45.979432 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-96b7ce93-145f-4270-bcb3-fd875fd36845 admin admin] The token cache is being invalidate because project 50b4838c797b4062bb2419d959ed9ecb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81552) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:45.979723 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-96b7ce93-145f-4270-bcb3-fd875fd36845 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:45.980354 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG keystone.notifications [None req-96b7ce93-145f-4270-bcb3-fd875fd36845 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81552) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:45.981825 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: [pid: 81552|app: 0|req: 1461/2922] 10.222.0.18 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:45 2022] DELETE /identity/v3/projects/50b4838c797b4062bb2419d959ed9ecb => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:55.612375 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:55.976515 dv-cinder-852209-10-iscsi devstack@keystone.service[81552]: DEBUG dbcounter [-] [81552] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81552) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:47:17.178728 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d77ad9-f26f-4461-9178-947cd7b0855f service neutron] Authenticating user token {{(pid=81551) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:17.180166 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-36d77ad9-f26f-4461-9178-947cd7b0855f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:17.248444 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d77ad9-f26f-4461-9178-947cd7b0855f service neutron] Validating token access rules against request {{(pid=81551) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:17.307261 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd32e2-b4dc-420e-a33e-5cca5230cd5e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ddfb151bbf844efae73ec1990e7eaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36ce80c4e0d64562a9ab3961be76fad9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81551) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:17.307785 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd32e2-b4dc-420e-a33e-5cca5230cd5e service placement] REQUEST_METHOD: `GET` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:17.308070 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd32e2-b4dc-420e-a33e-5cca5230cd5e service placement] SCRIPT_NAME: `/identity` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:17.308347 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd32e2-b4dc-420e-a33e-5cca5230cd5e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81551) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:17.310091 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.fernet_utils [None req-0bfd32e2-b4dc-420e-a33e-5cca5230cd5e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81551) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:17.321720 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd32e2-b4dc-420e-a33e-5cca5230cd5e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:17.323320 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd32e2-b4dc-420e-a33e-5cca5230cd5e service placement] RBAC: Authorization granted {{(pid=81551) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:17.333648 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: [pid: 81551|app: 0|req: 1462/2923] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Sep 1 16:47:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:47:27.328889 dv-cinder-852209-10-iscsi devstack@keystone.service[81551]: DEBUG dbcounter [-] [81551] Writing DB stats keystone:SELECT=58 {{(pid=81551) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}